2026-06-22 Mon 05:23:51:532.743 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: http-server-port 8091 2026-06-22 Mon 05:23:51:532.836 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Max-tasks 16 2026-06-22 Mon 05:23:51:532.843 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Max-slice 100 2026-06-22 Mon 05:23:51:532.848 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Collectors 10.0.0.38:8086 2026-06-22 Mon 05:23:51:532.852 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Collectors 10.0.0.50:8086 2026-06-22 Mon 05:23:51:532.856 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Collectors 10.0.0.33:8086 2026-06-22 Mon 05:23:51:532.878 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: ROLE : Generator 2026-06-22 Mon 05:23:51:532.883 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-22 Mon 05:23:51:532.887 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-6019-3. 2026-06-22 Mon 05:23:51:532.891 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-22 Mon 05:23:51:532.895 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-22 Mon 05:23:51:532.899 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-22 Mon 05:23:51:532.904 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-22 Mon 05:23:51:536.002 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: HTTP Introspect Init 2026-06-22 Mon 05:23:51:536.117 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-22 Mon 05:23:51:536.190 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Sandesh Http Server Port: 8091 2026-06-22 Mon 05:23:51:536.243 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-22 Mon 05:23:51:536.250 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-22 Mon 05:23:51:536.544 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Idle 2026-06-22 Mon 05:23:51:536.676 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: No Client: 1782105831536626 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:23:51:536.803 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-22 Mon 05:23:51:536.815 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-22 Mon 05:23:51:536.824 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-22 Mon 05:23:51:537.124 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvStart in state Idle 2026-06-22 Mon 05:23:51:537.265 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Disconnect 2026-06-22 Mon 05:23:51:537.367 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-22 Mon 05:23:51:537.728 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-22 Mon 05:23:51:537.863 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:23:51:537.916 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105831537256 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:23:51:537.928 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:23:51:537.939 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:23:51:538.078 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105831537351 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:23:51:538.127 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:23:51:538.154 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:23:51:538.229 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105831537441 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:23:51:538.246 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:23:51:538.256 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:23:51:538.347 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105831537467 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:23:51:538.405 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:23:51:538.417 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:23:51:538.527 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105831537847 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:23:51:538.550 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:23:51:540.750 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-22 Mon 05:23:51:540.770 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:23:51:540.825 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 0 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:23:51:540.895 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866321270336, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:23:51:540.927 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866321270336, Pid 1]: SANDESH: Send FAILED: 1782105831540803 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:23:51:540.936 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866321270336, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:23:56:543.173 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-22 Mon 05:23:56:543.209 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:23:56:543.229 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 1 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:24:01:545.234 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-22 Mon 05:24:01:545.293 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:24:01:545.315 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 2 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:24:06:546.772 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-22 Mon 05:24:06:546.819 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:24:06:546.846 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 3 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:24:11:591.598 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-22 Mon 05:24:11:591.721 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:24:11:591.742 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 4 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:24:16:629.148 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-22 Mon 05:24:16:629.210 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:24:16:629.235 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 5 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:24:21:714.761 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-22 Mon 05:24:21:714.829 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 6 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-22 Mon 05:24:26:752.409 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-22 Mon 05:24:31:758.778 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-22 Mon 05:24:31:758.824 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: -(-1):EvTcpConnectFail 2026-06-22 Mon 05:24:31:758.844 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2abf550 2026-06-22 Mon 05:24:31:758.849 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:758.861 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:758.870 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac4550 2026-06-22 Mon 05:24:31:758.874 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:758.880 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:758.889 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac5500 2026-06-22 Mon 05:24:31:758.893 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:758.909 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:758.919 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac4c00 2026-06-22 Mon 05:24:31:758.925 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:758.933 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:758.943 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac6020 2026-06-22 Mon 05:24:31:758.949 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:758.957 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:758.966 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac6b40 2026-06-22 Mon 05:24:31:758.972 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:758.980 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:758.989 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac7660 2026-06-22 Mon 05:24:31:758.995 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.002 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.012 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac8d00 2026-06-22 Mon 05:24:31:759.017 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.026 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.033 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac8220 2026-06-22 Mon 05:24:31:759.037 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.043 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.053 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2aca2c0 2026-06-22 Mon 05:24:31:759.058 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.066 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.076 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2acada0 2026-06-22 Mon 05:24:31:759.082 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.090 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.099 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2acb880 2026-06-22 Mon 05:24:31:759.104 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.112 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.121 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2ac97e0 2026-06-22 Mon 05:24:31:759.126 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.134 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.144 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2acc360 2026-06-22 Mon 05:24:31:759.149 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.157 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.167 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: RAC_Connect status: 0 0x2acce40 2026-06-22 Mon 05:24:31:759.173 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:24:31:759.178 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:24:31:759.383 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:24:31:759.411 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105871758597 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:24:31:759.422 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:31:759.431 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-22 Mon 05:24:31:759.436 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Connect : EvTcpConnectFail 2026-06-22 Mon 05:24:31:759.613 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:24:31:759.718 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Idle 2026-06-22 Mon 05:24:31:759.798 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:759.871 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105871759521 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 37, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:24:31:759.889 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:24:31:759.899 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:759.916 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105871759597 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:24:31:759.923 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:31:759.931 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:24:31:759.993 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:760.077 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105871759769 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 38, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:24:31:760.093 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:24:31:760.167 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:760.192 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105871760122 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782105871759716 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782105871759992 ] max_sm_queue_count = 5 ] 2026-06-22 Mon 05:24:31:760.198 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:24:31:760.206 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:760.343 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105871760154 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 38, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:24:31:760.365 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:24:31:760.719 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.740 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.757 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.764 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.775 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.781 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.795 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.801 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.810 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.816 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.826 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.832 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.845 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.854 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.867 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.877 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.890 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.897 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.906 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.912 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.922 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.928 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.946 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.956 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.962 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.970 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.977 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:760.986 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:24:31:760.994 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:24:31:761.048 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.071 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.082 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:761.134 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: SANDESH: Send FAILED: 1782105871761029 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:24:31:761.141 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:31:761.149 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:761.165 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: SANDESH: Send FAILED: 1782105871761067 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:24:31:761.175 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:31:761.091 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.557 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.576 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.604 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.623 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.646 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.646 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:761.684 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: SANDESH: Send FAILED: 1782105871761590 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:24:31:761.690 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:31:761.665 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.869 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.888 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:761.890 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866329667136, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:24:31:761.918 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866329667136, Pid 1]: SANDESH: Send FAILED: 1782105871761853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:24:31:761.925 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866329667136, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:31:761.900 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:31:762.091 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:24:32:759.855 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:24:32:760.232 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-22 Mon 05:24:32:760.352 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:24:32:760.383 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105872760050 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:24:32:760.390 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:24:32:760.398 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:24:32:760.505 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SANDESH: Send FAILED: 1782105872760333 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 83, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:24:32:760.531 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:24:32:762.734 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:34540::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:24:32:762.761 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: 10.0.0.33:34540::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:24:32:762.770 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: 10.0.0.33:34540::10.0.0.38:8086(-1):EvTcpConnected 2026-06-22 Mon 05:24:32:762.841 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:24:32:762.855 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:24:32:762.899 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: ClientInit 2026-06-22 Mon 05:24:32:762.918 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 1 2026-06-22 Mon 05:24:32:762.987 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:24:32:762.994 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:24:32:763.010 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: SANDESH: Send FAILED: 1782105872762890 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:24:32:763.019 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:24:32:763.029 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:24:32:763.036 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:24:32:763.058 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:24:32:763.066 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:24:32:763.138 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: SANDESH: Send FAILED: 1782105872762975 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:24:32:763.159 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866317071936, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:24:32:784.732 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: OnMessage control in state: ClientInit 2026-06-22 Mon 05:24:32:784.814 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-22 Mon 05:24:32:784.825 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-22 Mon 05:24:32:784.848 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Received Ctrl Message with size 0 2026-06-22 Mon 05:24:32:785.007 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782105871759716 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782105871759992 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 10 2026-06-22 Mon 05:24:32:785.050 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:24:32:785.080 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 11 2026-06-22 Mon 05:24:32:785.093 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:24:32:785.106 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:24:32:785.114 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:24:32:785.156 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Established 2026-06-22 Mon 05:24:39:698.483 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: OnMessage control in state: Established 2026-06-22 Mon 05:24:39:698.551 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-06-22 Mon 05:24:39:698.560 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Established : EvSandeshMessageRecv 2026-06-22 Mon 05:24:39:698.585 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: Received Ctrl Message with size 3 2026-06-22 Mon 05:24:39:698.598 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 11, total = 0 2026-06-22 Mon 05:24:39:698.608 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 13, total = 0 2026-06-22 Mon 05:24:39:698.619 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 1, total = 0 2026-06-22 Mon 05:24:39:698.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866428974656, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:25:01:859.981 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865333159488, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-22 Mon 05:26:01:860.575 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865337357888, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-22 Mon 05:27:01:861.321 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866433173056, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-22 Mon 05:27:37:063.600 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.33:58940 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-22 Mon 05:27:51:761.366 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866440315776, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.33:55216 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-22 Mon 05:28:01:862.110 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139865333159488, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-22 Mon 05:29:01:862.784 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866304476736, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-22 Mon 05:29:36:508.290 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Processing scm::EvStop in state Established 2026-06-22 Mon 05:29:36:508.375 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Idle 2026-06-22 Mon 05:29:36:508.484 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:29:36:508.496 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:29:36:508.592 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: SANDESH: Send FAILED: 1782106176508469 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782105831536554 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 280672 calls = 30 average_bytes = 9355 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 22 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 31, ] ] ] 2026-06-22 Mon 05:29:36:508.607 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139866420577856, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:29:52:533.815 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: http-server-port 8091 2026-06-22 Mon 05:29:52:533.905 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Max-tasks 16 2026-06-22 Mon 05:29:52:533.913 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Max-slice 100 2026-06-22 Mon 05:29:52:533.919 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Collectors 10.0.0.38:8086 2026-06-22 Mon 05:29:52:533.925 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Collectors 10.0.0.50:8086 2026-06-22 Mon 05:29:52:533.930 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Collectors 10.0.0.33:8086 2026-06-22 Mon 05:29:52:533.956 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: ROLE : Generator 2026-06-22 Mon 05:29:52:533.961 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-22 Mon 05:29:52:533.965 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-6019-3. 2026-06-22 Mon 05:29:52:533.969 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-22 Mon 05:29:52:533.980 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-22 Mon 05:29:52:533.984 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-22 Mon 05:29:52:533.989 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-22 Mon 05:29:52:548.945 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: HTTP Introspect Init 2026-06-22 Mon 05:29:52:549.082 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-22 Mon 05:29:52:549.172 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Sandesh Http Server Port: 8091 2026-06-22 Mon 05:29:52:549.219 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-22 Mon 05:29:52:549.227 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-22 Mon 05:29:52:549.573 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Idle 2026-06-22 Mon 05:29:52:549.705 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: No Client: 1782106192549652 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:29:52:549.834 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-22 Mon 05:29:52:549.848 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-22 Mon 05:29:52:549.864 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-22 Mon 05:29:52:556.640 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvStart in state Idle 2026-06-22 Mon 05:29:52:556.717 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Disconnect 2026-06-22 Mon 05:29:52:556.797 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-22 Mon 05:29:52:556.959 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-22 Mon 05:29:52:557.009 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:29:52:557.027 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106192556710 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:29:52:557.034 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:29:52:557.041 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:29:52:557.106 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106192556786 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:29:52:557.126 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:29:52:557.137 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:29:52:557.197 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106192556829 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:29:52:557.848 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:29:52:557.872 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:29:52:557.900 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106192556844 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:29:52:557.906 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:29:52:557.913 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:29:52:557.976 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106192556999 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:29:52:557.994 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:29:52:664.319 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-22 Mon 05:30:27:683.012 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:30:27:683.056 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c1590 2026-06-22 Mon 05:30:27:683.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.107 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.123 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c6cb0 2026-06-22 Mon 05:30:27:683.130 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.141 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.152 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c6550 2026-06-22 Mon 05:30:27:683.159 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.169 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.179 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c80f0 2026-06-22 Mon 05:30:27:683.186 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.197 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.207 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c75d0 2026-06-22 Mon 05:30:27:683.214 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.223 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.234 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c8c10 2026-06-22 Mon 05:30:27:683.241 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.251 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.261 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c9730 2026-06-22 Mon 05:30:27:683.276 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.286 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.296 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10ca2f0 2026-06-22 Mon 05:30:27:683.302 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.312 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.322 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cadd0 2026-06-22 Mon 05:30:27:683.329 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.339 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.351 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cb8b0 2026-06-22 Mon 05:30:27:683.358 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.368 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.378 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cc390 2026-06-22 Mon 05:30:27:683.384 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.394 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.404 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cce70 2026-06-22 Mon 05:30:27:683.410 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.420 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.431 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cd950 2026-06-22 Mon 05:30:27:683.437 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.447 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.068 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227682927 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:30:27:683.460 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10ce430 2026-06-22 Mon 05:30:27:683.498 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.508 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.520 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cef10 2026-06-22 Mon 05:30:27:683.489 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:27:683.527 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:30:27:683.553 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:30:27:683.579 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-22 Mon 05:30:27:683.588 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1):EvTcpConnectFail 2026-06-22 Mon 05:30:27:683.623 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-22 Mon 05:30:27:683.634 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : EvTcpConnectFail 2026-06-22 Mon 05:30:27:683.775 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:30:27:683.812 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Idle 2026-06-22 Mon 05:30:27:683.859 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:683.924 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227683706 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 17, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:30:27:683.942 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:30:27:683.949 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:683.962 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227683732 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:30:27:683.967 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:27:683.973 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:30:27:684.035 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:684.108 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227683848 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:30:27:684.124 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:30:27:684.189 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:684.218 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.245 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.263 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.274 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.291 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.302 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.318 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.329 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.344 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.356 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.371 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.382 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.404 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.415 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.430 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.441 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.456 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.467 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:684.482 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:684.224 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227684139 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106227683811 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106227684034 ] max_sm_queue_count = 5 ] 2026-06-22 Mon 05:30:27:684.520 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:30:27:684.532 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:684.629 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227684172 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 9, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:30:27:684.647 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:30:27:684.502 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:693.595 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.621 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.632 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.643 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.653 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.679 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.690 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.699 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.722 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.732 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.828 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:693.842 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:693.857 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:693.864 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:693.874 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:693.880 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:693.891 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:693.909 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:693.919 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:30:27:693.925 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:30:27:693.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.956 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.965 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.973 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:693.981 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:30:27:695.827 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:695.879 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227693561 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:30:27:695.888 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:27:695.896 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:695.909 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227693616 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:30:27:695.924 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:27:695.930 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:695.942 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227693665 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:30:27:695.946 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:27:695.952 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:30:27:695.962 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106227693951 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:30:27:695.967 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:28:683.927 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:30:28:684.185 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-22 Mon 05:30:28:684.290 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:30:28:684.322 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106228683990 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:30:28:684.338 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:30:28:684.348 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:30:28:684.462 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106228684272 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 63, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 15, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:30:28:684.483 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:30:28:685.782 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:52722::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:30:28:685.809 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:52722::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:30:28:685.816 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:52722::10.0.0.38:8086(-1):EvTcpConnected 2026-06-22 Mon 05:30:28:685.872 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:30:28:685.895 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:30:28:685.932 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: ClientInit 2026-06-22 Mon 05:30:28:685.950 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 1 2026-06-22 Mon 05:30:28:686.017 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:30:28:687.981 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:30:28:688.030 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: SANDESH: Send FAILED: 1782106228685926 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:30:28:688.044 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:30:28:688.063 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:30:28:688.068 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:30:28:688.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:30:28:688.094 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:30:28:688.169 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: SANDESH: Send FAILED: 1782106228686007 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 64, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 17, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:30:28:688.240 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:30:28:690.771 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: OnMessage control in state: ClientInit 2026-06-22 Mon 05:30:28:690.823 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-22 Mon 05:30:28:690.833 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-22 Mon 05:30:28:690.857 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Received Ctrl Message with size 0 2026-06-22 Mon 05:30:28:690.995 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106227683811 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106227684034 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 64, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 17, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 10 2026-06-22 Mon 05:30:28:691.023 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:30:28:691.042 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 10 2026-06-22 Mon 05:30:28:691.054 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:30:28:691.060 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:30:28:691.066 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:30:28:691.095 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Established 2026-06-22 Mon 05:30:57:684.425 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199610807872, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-22 Mon 05:31:50:309.245 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c75d0 2026-06-22 Mon 05:31:50:309.319 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:309.386 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c6550 2026-06-22 Mon 05:31:50:309.397 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:309.533 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c80f0 2026-06-22 Mon 05:31:50:309.546 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:309.571 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-22 Mon 05:31:50:309.579 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c1590 2026-06-22 Mon 05:31:50:309.586 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:309.606 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c6cb0 2026-06-22 Mon 05:31:50:309.615 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:309.818 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:31:50:309.877 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:31:50:309.910 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:31:50:309.967 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:31:50:309.980 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:31:50:322.433 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:52722::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-22 Mon 05:31:50:322.468 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:52722::10.0.0.38:8086(-1):EvTcpClose 2026-06-22 Mon 05:31:50:322.597 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvTcpClose in state Established 2026-06-22 Mon 05:31:50:322.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Established : EvTcpClose 2026-06-22 Mon 05:31:50:322.910 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-22 Mon 05:31:50:322.980 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:50:323.084 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SANDESH: Send FAILED: 1782106310322717 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = cn-jenkins-deploy-platform-ansible-os-6019-1. collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 94361 calls = 12 average_bytes = 7863 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 70, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-22 Mon 05:31:50:323.108 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:323.118 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:50:323.133 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SANDESH: Send FAILED: 1782106310322750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:31:50:323.139 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:50:323.145 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-22 Mon 05:31:50:323.174 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:50:323.243 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SANDESH: Send FAILED: 1782106310322968 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 94361 calls = 12 average_bytes = 7863 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 70, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-22 Mon 05:31:50:323.256 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:325.384 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:47086::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:31:50:325.412 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:47086::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:31:50:325.420 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:47086::10.0.0.33:8086(-1):EvTcpConnected 2026-06-22 Mon 05:31:50:325.481 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:31:50:325.506 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:31:50:325.538 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit 2026-06-22 Mon 05:31:50:325.558 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 2 2026-06-22 Mon 05:31:50:325.626 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:50:325.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:31:50:325.651 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106310325532 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:31:50:325.657 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:31:50:325.663 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:50:325.668 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:31:50:325.677 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:50:325.682 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:325.787 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106310325614 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 94361 calls = 12 average_bytes = 7863 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 71, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 22, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-22 Mon 05:31:50:325.807 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:331.228 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10ca2f0 2026-06-22 Mon 05:31:50:331.278 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:331.323 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10cadd0 2026-06-22 Mon 05:31:50:331.332 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:331.351 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10cb8b0 2026-06-22 Mon 05:31:50:331.359 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:331.378 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c9730 2026-06-22 Mon 05:31:50:331.385 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:331.410 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-22 Mon 05:31:50:331.439 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10c8c10 2026-06-22 Mon 05:31:50:331.446 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:333.664 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:50:333.715 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:31:50:333.743 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SANDESH: Send FAILED: 1782106310331515 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:31:50:333.750 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:31:50:337.870 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:31:50:337.931 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:31:50:337.948 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:31:50:337.961 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:31:50:337.973 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:31:50:393.005 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: OnMessage control in state: ClientInit 2026-06-22 Mon 05:31:50:393.059 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-22 Mon 05:31:50:393.075 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-22 Mon 05:31:50:393.091 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Received Ctrl Message with size 0 2026-06-22 Mon 05:31:50:393.231 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 94361 calls = 12 average_bytes = 7863 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 35 dequeues = 35 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Established state_since = 1782106228691094 last_state = Established last_event = scm::EvSandeshSend last_event_at = 1782106257684489 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 9 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 66185 calls = 8 average_bytes = 8273 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 71, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 22, _iter59->first = sent _iter59->second = 13, ] ] max_sm_queue_count = 5 proxy seq 18 2026-06-22 Mon 05:31:50:393.266 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:31:50:393.284 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 16 2026-06-22 Mon 05:31:50:393.296 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:31:50:393.323 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.121306 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 2 2026-06-22 Mon 05:31:50:393.332 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:31:50:393.338 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:31:50:393.378 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Established 2026-06-22 Mon 05:31:50:437.724 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10cef10 2026-06-22 Mon 05:31:50:437.791 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:437.960 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10ce430 2026-06-22 Mon 05:31:50:437.972 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:437.999 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10cd950 2026-06-22 Mon 05:31:50:438.007 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:438.037 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10cce70 2026-06-22 Mon 05:31:50:438.048 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:438.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-22 Mon 05:31:50:438.097 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Disconnect status: -1 0x10cc390 2026-06-22 Mon 05:31:50:438.107 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-22 Mon 05:31:50:438.333 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:31:50:438.356 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:31:50:438.366 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:31:50:438.376 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:31:50:438.386 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:31:50:457.969 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:47086::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-22 Mon 05:31:50:458.003 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:47086::10.0.0.33:8086(-1):EvTcpClose 2026-06-22 Mon 05:31:50:458.060 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvTcpClose in state Established 2026-06-22 Mon 05:31:50:458.073 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Established : EvTcpClose 2026-06-22 Mon 05:31:50:458.381 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-22 Mon 05:31:50:458.472 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:50:458.622 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106310458168 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = cn-jenkins-deploy-platform-ansible-os-6019-3. collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 147769 calls = 17 average_bytes = 8692 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 83, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 21, ] ] ] 2026-06-22 Mon 05:31:50:458.659 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:458.669 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:50:458.699 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106310458207 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:50:458.714 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:50:458.722 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-22 Mon 05:31:50:458.759 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:50:458.856 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106310458457 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 147769 calls = 17 average_bytes = 8692 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 83, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 21, ] ] ] 2026-06-22 Mon 05:31:50:458.874 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:459.507 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-22 Mon 05:31:50:459.519 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1):EvTcpConnectFail 2026-06-22 Mon 05:31:50:459.549 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-22 Mon 05:31:50:459.558 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Connect : EvTcpConnectFail 2026-06-22 Mon 05:31:50:459.668 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:31:50:459.701 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Idle 2026-06-22 Mon 05:31:50:460.686 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:50:460.801 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: SANDESH: Send FAILED: 1782106310459605 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 147769 calls = 17 average_bytes = 8692 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 83, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 21, ] ] ] 2026-06-22 Mon 05:31:50:460.836 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:50:460.848 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:50:460.872 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: SANDESH: Send FAILED: 1782106310459647 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:50:460.879 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:50:460.887 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:31:50:460.915 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:50:460.984 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: SANDESH: Send FAILED: 1782106310460663 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 147769 calls = 17 average_bytes = 8692 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 21, ] ] ] 2026-06-22 Mon 05:31:50:461.001 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:51:459.905 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:31:51:460.223 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-22 Mon 05:31:51:460.318 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:51:460.342 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106311460030 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:51:460.350 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:51:460.358 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:51:460.470 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106311460304 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 147769 calls = 17 average_bytes = 8692 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 21, ] ] ] 2026-06-22 Mon 05:31:51:460.497 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:51:462.092 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:38644::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:31:51:462.116 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38644::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:31:51:462.125 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38644::10.0.0.38:8086(-1):EvTcpConnected 2026-06-22 Mon 05:31:51:462.185 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:31:51:462.216 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:31:51:462.254 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: ClientInit 2026-06-22 Mon 05:31:51:462.276 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 3 2026-06-22 Mon 05:31:51:462.351 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:51:462.359 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:31:51:462.377 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106311462245 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:51:462.384 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:31:51:462.390 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:51:462.395 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:31:51:462.414 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:51:462.419 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:31:51:462.504 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106311462340 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 147769 calls = 17 average_bytes = 8692 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 21, ] ] ] 2026-06-22 Mon 05:31:51:462.517 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:31:51:466.669 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38644::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-22 Mon 05:31:51:466.697 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38644::10.0.0.38:8086(-1):EvTcpClose 2026-06-22 Mon 05:31:51:466.982 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-22 Mon 05:31:51:467.001 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: ClientInit : EvTcpClose 2026-06-22 Mon 05:31:51:467.109 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Return to idle with 1000 2026-06-22 Mon 05:31:51:467.117 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Idle 2026-06-22 Mon 05:31:51:467.157 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:51:467.259 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: SANDESH: Send FAILED: 1782106311467064 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 149342 calls = 18 average_bytes = 8296 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 22, ] ] ] 2026-06-22 Mon 05:31:51:467.275 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:51:467.282 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:51:467.296 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: SANDESH: Send FAILED: 1782106311467093 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:51:467.302 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:51:467.309 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:31:51:467.353 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:51:467.442 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: SANDESH: Send FAILED: 1782106311467148 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 3 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 149342 calls = 18 average_bytes = 8296 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 22, ] ] ] 2026-06-22 Mon 05:31:51:467.457 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:52:467.330 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:31:52:467.744 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-22 Mon 05:31:52:467.920 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:52:467.958 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: SANDESH: Send FAILED: 1782106312467427 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:52:467.986 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:52:467.997 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:52:468.148 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: SANDESH: Send FAILED: 1782106312467899 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 3 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 149342 calls = 18 average_bytes = 8296 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 38, _iter59->first = sent _iter59->second = 22, ] ] ] 2026-06-22 Mon 05:31:52:468.168 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:52:469.478 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:47108::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:31:52:469.524 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:47108::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:31:52:469.532 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:47108::10.0.0.33:8086(-1):EvTcpConnected 2026-06-22 Mon 05:31:52:469.615 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:31:52:469.823 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:31:52:469.882 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit 2026-06-22 Mon 05:31:52:469.916 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 4 2026-06-22 Mon 05:31:52:470.033 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:52:470.049 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:31:52:470.077 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106312469870 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:52:470.085 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:31:52:470.093 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:52:470.098 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:31:52:470.110 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:52:470.116 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:31:52:470.202 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: SANDESH: Send FAILED: 1782106312470015 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 149342 calls = 18 average_bytes = 8296 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 22, ] ] ] 2026-06-22 Mon 05:31:52:470.224 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200359736896, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:31:55:309.911 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:31:55:310.075 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:31:55:310.125 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:31:55:310.162 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:31:55:310.201 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:31:55:310.485 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:31:55:310.527 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:31:55:310.547 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:31:55:310.562 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:31:55:310.578 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:31:55:337.960 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:31:55:338.090 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:31:55:338.133 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:31:55:338.168 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:31:55:338.202 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:31:55:338.311 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:31:55:338.347 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:31:55:338.365 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:31:55:338.386 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:31:55:338.400 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:31:55:438.404 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:31:55:438.553 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:31:55:438.587 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:31:55:438.626 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:31:55:438.646 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:31:55:438.661 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:31:55:438.672 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:31:55:438.715 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:31:55:438.750 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:31:55:438.764 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:31:56:470.410 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: 10.0.0.33:8086 EvConnectTimerExpired in state ClientInit 2026-06-22 Mon 05:31:56:470.477 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-06-22 Mon 05:31:56:470.489 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: ClientInit : EvConnectTimerExpired 2026-06-22 Mon 05:31:56:470.653 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Return to idle with 1000 2026-06-22 Mon 05:31:56:470.663 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Idle 2026-06-22 Mon 05:31:56:470.713 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:56:470.841 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106316470559 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 150915 calls = 19 average_bytes = 7942 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 23, ] ] ] 2026-06-22 Mon 05:31:56:470.869 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:56:470.882 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:56:470.903 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106316470592 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:56:470.910 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:56:470.917 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:31:56:470.924 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:56:470.993 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106316470694 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 4 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 150915 calls = 19 average_bytes = 7942 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 23, ] ] ] 2026-06-22 Mon 05:31:56:471.007 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:470.836 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:31:57:471.170 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-22 Mon 05:31:57:471.262 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:57:471.288 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106317470935 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:57:471.297 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:57:471.307 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:57:471.406 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106317471248 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 4 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 150915 calls = 19 average_bytes = 7942 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 23, ] ] ] 2026-06-22 Mon 05:31:57:471.430 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:472.976 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:57746::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:31:57:473.005 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:57746::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:31:57:473.015 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:57746::10.0.0.50:8086(-1):EvTcpConnected 2026-06-22 Mon 05:31:57:473.081 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:31:57:473.093 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:31:57:473.136 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: ClientInit 2026-06-22 Mon 05:31:57:473.150 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 5 2026-06-22 Mon 05:31:57:473.221 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:57:473.228 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:31:57:473.243 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106317473128 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:57:473.257 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:31:57:473.264 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:57:473.269 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:31:57:473.278 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:57:473.282 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:473.400 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106317473210 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 150915 calls = 19 average_bytes = 7942 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 23, ] ] ] 2026-06-22 Mon 05:31:57:473.425 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:477.617 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:57746::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-22 Mon 05:31:57:477.650 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:57746::10.0.0.50:8086(-1):EvTcpClose 2026-06-22 Mon 05:31:57:477.836 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-22 Mon 05:31:57:477.869 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: ClientInit : EvTcpClose 2026-06-22 Mon 05:31:57:477.993 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Return to idle with 1000 2026-06-22 Mon 05:31:57:478.003 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Idle 2026-06-22 Mon 05:31:57:478.062 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:57:478.197 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: SANDESH: Send FAILED: 1782106317477951 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 152488 calls = 20 average_bytes = 7624 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 24, ] ] ] 2026-06-22 Mon 05:31:57:478.225 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:478.236 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:57:478.257 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: SANDESH: Send FAILED: 1782106317477982 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:57:478.271 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:57:478.281 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:31:57:478.315 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:57:478.436 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: SANDESH: Send FAILED: 1782106317478048 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 152488 calls = 20 average_bytes = 7624 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 24, ] ] ] 2026-06-22 Mon 05:31:57:478.454 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:685.062 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-22 Mon 05:31:57:685.137 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:57:685.180 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: SANDESH: Send FAILED: 1782106317684921 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 76 dequeues = 76 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 103 dequeues = 103 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106317478002 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106317478314 ] max_sm_queue_count = 6 ] 2026-06-22 Mon 05:31:57:685.190 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:57:685.197 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:31:57:685.297 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: SANDESH: Send FAILED: 1782106317685039 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 152488 calls = 20 average_bytes = 7624 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 24, ] ] ] 2026-06-22 Mon 05:31:57:685.315 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200355538496, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:58:478.312 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:31:58:478.576 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-22 Mon 05:31:58:478.710 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:58:478.743 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106318478406 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:58:478.753 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:31:58:478.762 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:31:58:478.916 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106318478666 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 152488 calls = 20 average_bytes = 7624 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 24, ] ] ] 2026-06-22 Mon 05:31:58:478.951 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:31:58:481.013 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:38672::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:31:58:481.040 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38672::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:31:58:481.048 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38672::10.0.0.38:8086(-1):EvTcpConnected 2026-06-22 Mon 05:31:58:481.456 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:31:58:481.474 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:31:58:481.519 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: ClientInit 2026-06-22 Mon 05:31:58:481.537 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 6 2026-06-22 Mon 05:31:58:481.634 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:58:481.646 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:31:58:481.671 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106318481512 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:31:58:481.680 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:31:58:481.687 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:58:481.717 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:31:58:481.735 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:31:58:481.741 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:31:58:481.874 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106318481618 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 152488 calls = 20 average_bytes = 7624 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 88, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 56, _iter59->first = sent _iter59->second = 24, ] ] ] 2026-06-22 Mon 05:31:58:481.896 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:32:00:310.595 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:00:310.836 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:00:310.876 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:00:310.909 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:00:310.941 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:00:311.159 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:00:311.206 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:00:311.227 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:00:320.092 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:00:320.178 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:00:338.411 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:00:338.595 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:00:338.640 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:00:338.675 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:00:338.728 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:00:338.949 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:00:339.000 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:00:339.016 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:00:339.038 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:00:339.057 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:00:438.723 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:00:438.920 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:00:438.967 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:00:439.016 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:00:439.045 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:00:439.062 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:00:439.075 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:00:439.105 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:00:439.132 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:00:439.145 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:01:603.105 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.33:38672::10.0.0.38:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-22 Mon 05:32:01:603.177 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.33:38672::10.0.0.38:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:32:01:603.206 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38672::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-22 Mon 05:32:01:603.216 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38672::10.0.0.38:8086(-1):EvTcpClose 2026-06-22 Mon 05:32:01:603.282 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-22 Mon 05:32:01:603.293 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: ClientInit : EvTcpClose 2026-06-22 Mon 05:32:01:603.423 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Return to idle with 1000 2026-06-22 Mon 05:32:01:603.435 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Idle 2026-06-22 Mon 05:32:01:603.488 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:01:603.673 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: SANDESH: Send FAILED: 1782106321603365 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:01:603.710 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:01:603.729 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:01:603.779 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: SANDESH: Send FAILED: 1782106321603400 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:32:01:603.790 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:01:603.799 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:32:01:603.844 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:01:603.933 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: SANDESH: Send FAILED: 1782106321603477 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:01:603.951 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199602411072, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:02:606.084 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:32:02:606.325 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-22 Mon 05:32:02:606.431 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:32:02:606.461 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: SANDESH: Send FAILED: 1782106322606153 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:32:02:606.471 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:02:606.481 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:32:02:606.589 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: SANDESH: Send FAILED: 1782106322606416 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:02:606.618 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200411915840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:02:607.287 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-22 Mon 05:32:02:607.304 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1):EvTcpConnectFail 2026-06-22 Mon 05:32:02:608.845 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-22 Mon 05:32:02:608.867 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Connect : EvTcpConnectFail 2026-06-22 Mon 05:32:02:609.018 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:32:02:609.054 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Idle 2026-06-22 Mon 05:32:02:609.120 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:02:609.270 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: SANDESH: Send FAILED: 1782106322608953 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:02:609.293 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:02:609.304 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:02:609.324 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: SANDESH: Send FAILED: 1782106322608991 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:32:02:609.331 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:02:609.341 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:32:02:609.350 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:02:609.478 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: SANDESH: Send FAILED: 1782106322609106 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 91, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:02:609.495 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:05:311.260 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:05:311.436 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:05:311.499 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:05:311.842 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:05:311.888 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:05:311.906 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:05:320.211 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:05:320.389 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:05:321.473 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:05:321.727 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:05:339.047 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:05:339.183 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:05:339.233 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:05:339.263 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:05:339.294 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:05:339.481 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:05:339.509 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:05:339.521 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:05:339.533 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:05:339.546 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:05:439.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:05:439.256 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:05:439.321 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:05:439.348 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:05:439.364 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:05:439.399 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:05:439.424 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:05:439.453 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:05:439.466 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:05:439.476 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:10:311.924 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:10:312.105 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:10:312.157 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:10:312.500 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:10:312.542 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:10:312.556 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:10:321.557 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:10:324.646 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:10:324.945 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:10:324.982 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:10:339.564 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:10:342.423 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:10:342.490 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:10:342.598 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:10:342.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:10:342.803 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:10:342.835 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:10:342.880 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:10:342.893 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:10:342.904 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:10:439.415 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:10:442.075 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:10:442.165 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:10:442.198 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:10:442.211 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:10:442.248 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:10:442.281 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:10:442.322 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:10:442.340 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:10:442.356 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:15:312.594 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:15:312.858 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:15:312.902 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:15:313.185 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:15:313.247 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:15:313.264 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:15:325.039 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:15:325.231 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:15:325.580 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:15:325.638 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:15:342.897 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:15:343.091 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:15:343.131 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:15:343.171 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:15:343.195 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:15:343.360 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:15:343.422 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:15:343.447 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:15:343.460 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:15:343.484 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:15:442.262 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:15:442.554 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:15:442.623 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:15:442.651 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:15:442.664 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:15:442.712 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:15:442.752 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:15:442.794 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:15:442.864 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:15:442.879 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:20:313.294 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:20:313.460 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:20:313.494 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:20:313.860 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:20:313.898 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:20:313.910 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:20:325.697 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:20:325.877 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:20:326.265 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:20:326.338 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:20:343.564 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:20:343.870 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:20:343.925 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:20:343.960 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:20:343.994 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:20:344.282 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:20:344.341 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:20:344.358 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:20:344.374 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:20:344.390 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:20:442.753 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:20:442.998 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:20:443.064 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:20:443.093 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:20:443.103 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:20:443.136 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:20:443.159 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:20:443.187 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:20:443.199 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:20:443.213 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:25:313.950 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:25:314.155 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:25:314.213 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:25:314.401 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:25:314.440 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:25:324.238 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:25:326.371 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:25:326.551 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:25:326.812 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:25:326.850 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:25:344.394 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:25:344.574 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:25:344.641 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:25:344.680 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:25:344.754 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:25:344.924 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:25:344.960 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:25:344.977 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:25:344.993 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:25:345.007 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:25:443.152 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:25:443.349 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:25:443.421 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:25:443.453 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:25:443.466 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:25:443.506 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:25:443.530 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:25:443.561 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:25:443.574 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:25:443.584 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:27:685.470 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200363935296, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:27:685.567 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200363935296, Pid 1]: SANDESH: Send FAILED: 1782106347685310 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 91 dequeues = 91 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 125 dequeues = 125 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106322609053 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106322609349 ] max_sm_queue_count = 6 ] 2026-06-22 Mon 05:32:27:685.582 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200363935296, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:27:685.591 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200363935296, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:27:685.744 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200363935296, Pid 1]: SANDESH: Send FAILED: 1782106347685399 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 91, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:27:685.779 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200363935296, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:30:314.507 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:30:314.683 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:30:314.943 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:30:314.980 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:30:324.366 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:30:324.798 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:30:326.872 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:30:326.962 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:30:327.139 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:30:327.167 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:30:345.009 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:30:345.149 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:30:345.193 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:30:345.222 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:30:345.254 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:30:345.451 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:30:345.483 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:30:345.498 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:30:345.510 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:30:345.522 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:30:443.515 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:30:443.708 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:30:443.779 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:30:443.814 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:30:443.824 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:30:443.854 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:30:443.878 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:30:443.906 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:30:443.918 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:30:443.928 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:32:610.250 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:32:32:610.453 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-22 Mon 05:32:32:610.537 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:32:32:610.558 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106352610324 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:32:32:610.565 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:32:610.572 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:32:32:610.704 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106352610524 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 91, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 68, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:32:610.743 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:32:611.076 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-22 Mon 05:32:32:611.087 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1):EvTcpConnectFail 2026-06-22 Mon 05:32:32:611.130 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-22 Mon 05:32:32:611.153 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Connect : EvTcpConnectFail 2026-06-22 Mon 05:32:32:611.242 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:32:32:611.267 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Idle 2026-06-22 Mon 05:32:32:611.306 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:32:611.424 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106352611204 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 91, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 70, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:32:611.444 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:32:611.452 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:32:611.466 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106352611226 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:32:32:611.472 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:32:611.478 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:32:32:611.498 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:32:611.597 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: SANDESH: Send FAILED: 1782106352611297 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 92, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 70, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:32:611.614 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199476590144, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:32:685.137 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200397784640, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:32:685.248 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200397784640, Pid 1]: SANDESH: Send FAILED: 1782106352685048 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.121306 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-06-22 Mon 05:32:32:685.263 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200397784640, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-06-22 Mon 05:32:35:315.077 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:35:315.222 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:35:315.515 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:35:315.546 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:35:324.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:35:325.272 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:35:327.186 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:35:327.268 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:35:327.489 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:35:327.511 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:35:345.535 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:35:345.710 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:35:345.760 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:35:345.805 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:35:345.834 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:35:345.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:35:345.971 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:35:345.985 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:35:345.996 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:35:346.020 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:35:443.858 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:35:444.075 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:35:444.145 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:35:444.177 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:35:444.188 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:35:444.224 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:35:444.257 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:35:444.289 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:35:444.307 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:35:444.321 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:40:315.605 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:40:315.816 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:40:316.141 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:40:316.170 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:40:325.356 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:40:325.762 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:40:327.536 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:40:327.622 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:40:327.880 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:40:327.915 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:40:346.028 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:40:346.180 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:40:346.215 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:40:346.240 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:40:346.268 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:40:346.477 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:40:346.500 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:40:346.512 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:40:346.522 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:40:346.533 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:40:445.140 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:40:445.393 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:40:445.458 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:40:445.498 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:40:445.534 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:40:445.576 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:40:445.599 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:40:445.610 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:40:445.620 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:40:445.643 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:45:316.230 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:45:316.478 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:45:316.790 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:45:316.827 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:45:325.862 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:45:326.297 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:45:327.932 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:45:328.009 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:45:328.254 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:45:328.301 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:45:346.559 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:45:346.757 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:45:346.816 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:45:346.878 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:45:346.914 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:45:347.087 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:45:347.124 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:45:347.141 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:45:347.156 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:45:347.254 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:45:445.692 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:45:445.853 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:45:445.885 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:45:445.911 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:45:445.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:45:445.983 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:45:446.007 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:45:446.018 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:45:446.028 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:45:446.038 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:50:316.879 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:50:317.124 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:50:317.438 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6550 2026-06-22 Mon 05:32:50:317.466 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c75d0 2026-06-22 Mon 05:32:50:326.401 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:50:326.810 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c80f0 2026-06-22 Mon 05:32:50:328.326 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:50:328.445 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:50:328.761 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c6cb0 2026-06-22 Mon 05:32:50:328.813 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c1590 2026-06-22 Mon 05:32:50:347.195 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:50:347.333 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:50:347.379 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:50:347.418 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:50:347.455 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:50:347.527 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ca2f0 2026-06-22 Mon 05:32:50:347.573 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cb8b0 2026-06-22 Mon 05:32:50:347.586 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c8c10 2026-06-22 Mon 05:32:50:347.598 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cadd0 2026-06-22 Mon 05:32:50:347.630 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10c9730 2026-06-22 Mon 05:32:50:446.071 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:50:446.274 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:50:446.332 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:50:446.396 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:50:446.452 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cef10 2026-06-22 Mon 05:32:50:446.484 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10ce430 2026-06-22 Mon 05:32:50:446.501 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cd950 2026-06-22 Mon 05:32:50:446.516 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cce70 2026-06-22 Mon 05:32:50:446.527 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:50:446.565 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: -1 0x10cc390 2026-06-22 Mon 05:32:55:317.519 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6550 2026-06-22 Mon 05:32:55:317.742 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c75d0 2026-06-22 Mon 05:32:55:318.025 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c6550 2026-06-22 Mon 05:32:55:318.045 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:318.072 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:318.100 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c75d0 2026-06-22 Mon 05:32:55:318.108 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:318.118 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:318.362 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:318.382 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:318.399 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:318.412 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:318.438 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:319.440 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:55:319.488 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106375318492 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:32:55:319.499 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:55:319.526 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:326.876 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c80f0 2026-06-22 Mon 05:32:55:327.240 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c80f0 2026-06-22 Mon 05:32:55:327.253 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:327.273 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:327.550 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:327.582 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:327.610 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:328.830 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c6cb0 2026-06-22 Mon 05:32:55:328.931 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c1590 2026-06-22 Mon 05:32:55:329.152 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c6cb0 2026-06-22 Mon 05:32:55:329.168 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:329.184 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:329.201 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c1590 2026-06-22 Mon 05:32:55:329.205 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:329.226 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:329.441 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:329.457 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:329.482 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:329.494 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:329.500 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:329.511 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:347.628 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ca2f0 2026-06-22 Mon 05:32:55:347.790 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cb8b0 2026-06-22 Mon 05:32:55:347.833 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c8c10 2026-06-22 Mon 05:32:55:347.864 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cadd0 2026-06-22 Mon 05:32:55:347.898 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10c9730 2026-06-22 Mon 05:32:55:347.975 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10ca2f0 2026-06-22 Mon 05:32:55:347.982 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:348.001 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:348.023 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cb8b0 2026-06-22 Mon 05:32:55:348.027 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:348.034 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:348.040 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c8c10 2026-06-22 Mon 05:32:55:348.044 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:348.050 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:348.095 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cadd0 2026-06-22 Mon 05:32:55:348.100 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:348.106 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:348.113 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10c9730 2026-06-22 Mon 05:32:55:348.118 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:348.126 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:348.199 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:348.211 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:348.236 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:348.307 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:348.319 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:349.035 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200397784640, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:55:349.080 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200397784640, Pid 1]: SANDESH: Send FAILED: 1782106375348359 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:32:55:349.090 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200397784640, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:55:349.109 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:349.132 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:349.142 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:349.168 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:349.181 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:349.188 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:349.198 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:349.204 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:349.218 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:349.231 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:446.532 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cef10 2026-06-22 Mon 05:32:55:446.709 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10ce430 2026-06-22 Mon 05:32:55:446.783 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cef10 2026-06-22 Mon 05:32:55:446.791 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:446.815 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:446.827 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10ce430 2026-06-22 Mon 05:32:55:446.833 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:446.842 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:446.850 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cd950 2026-06-22 Mon 05:32:55:446.894 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cce70 2026-06-22 Mon 05:32:55:446.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x10cc390 2026-06-22 Mon 05:32:55:447.025 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cd950 2026-06-22 Mon 05:32:55:447.032 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:447.043 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:447.063 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cce70 2026-06-22 Mon 05:32:55:447.069 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:447.077 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:447.085 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: RAC_Connect status: 0 0x10cc390 2026-06-22 Mon 05:32:55:447.091 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:32:55:447.099 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-22 Mon 05:32:55:447.689 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:447.711 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:447.725 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:447.734 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:447.747 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:447.770 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:447.785 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:447.794 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:447.806 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-22 Mon 05:32:55:447.815 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-22 Mon 05:32:55:447.837 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:456.170 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:55:456.239 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: SANDESH: Send FAILED: 1782106375447867 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:32:55:456.259 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200488576576, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:32:55:456.285 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:456.306 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:456.315 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:55:456.323 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-22 Mon 05:32:57:686.969 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-22 Mon 05:32:57:687.027 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:57:687.070 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: SANDESH: Send FAILED: 1782106377686850 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 102 dequeues = 102 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 139 dequeues = 139 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106352611266 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106375456155 ] max_sm_queue_count = 6 ] 2026-06-22 Mon 05:32:57:687.081 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:32:57:687.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:32:57:687.199 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: SANDESH: Send FAILED: 1782106377686948 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 152, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 77, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:32:57:687.216 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:02:611.388 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:33:02:611.599 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-22 Mon 05:33:02:611.715 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:33:02:611.740 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106382611466 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:33:02:611.782 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:33:02:611.794 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:33:02:611.867 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-22 Mon 05:33:02:611.890 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: -(-1):EvTcpConnectFail 2026-06-22 Mon 05:33:02:611.920 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106382611665 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 152, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 79, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:33:02:611.936 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:02:611.947 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-22 Mon 05:33:02:611.955 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : EvTcpConnectFail 2026-06-22 Mon 05:33:02:612.053 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-22 Mon 05:33:02:612.078 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Idle 2026-06-22 Mon 05:33:02:612.121 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:33:02:612.204 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106382612016 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 152, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 81, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:33:02:612.217 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:02:612.224 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:33:02:612.237 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106382612035 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:33:02:612.242 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:33:02:612.251 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:33:02:612.272 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:33:02:612.340 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106382612113 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 153, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 81, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:33:02:612.351 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:27:687.258 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:33:27:687.333 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106407687145 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 109 dequeues = 109 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 149 dequeues = 149 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106382612077 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106382612271 ] max_sm_queue_count = 6 ] 2026-06-22 Mon 05:33:27:687.355 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:27:687.362 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:33:27:687.445 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: SANDESH: Send FAILED: 1782106407687235 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 153, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 84, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:33:27:687.456 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200416114240, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:32:613.248 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-22 Mon 05:33:32:613.496 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-22 Mon 05:33:32:613.636 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:33:32:613.680 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106412613323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:33:32:613.691 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:33:32:613.707 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:33:32:613.836 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: SANDESH: Send FAILED: 1782106412613618 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 153, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 86, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:33:32:613.854 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200393586240, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:33:32:615.246 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:38428::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-22 Mon 05:33:32:615.262 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38428::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:33:32:615.269 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200499917696, Pid 1]: 10.0.0.33:38428::10.0.0.33:8086(-1):EvTcpConnected 2026-06-22 Mon 05:33:32:616.446 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:33:32:616.471 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:33:32:616.513 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: ClientInit 2026-06-22 Mon 05:33:32:616.524 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 7 2026-06-22 Mon 05:33:32:616.598 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:33:32:616.605 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:33:32:616.627 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: SANDESH: Send FAILED: 1782106412616505 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:33:32:616.634 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:33:32:616.640 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:33:32:616.645 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:33:32:616.665 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:33:32:616.716 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:33:32:616.822 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: SANDESH: Send FAILED: 1782106412616588 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 154, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 25, ] ] ] 2026-06-22 Mon 05:33:32:616.843 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200492774976, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:33:32:628.267 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: OnMessage control in state: ClientInit 2026-06-22 Mon 05:33:32:628.321 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-22 Mon 05:33:32:628.329 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-22 Mon 05:33:32:628.346 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Received Ctrl Message with size 0 2026-06-22 Mon 05:33:32:628.495 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 154061 calls = 21 average_bytes = 7336 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 109 dequeues = 109 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 149 dequeues = 149 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1782106382612077 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1782106382612271 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 9 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 66185 calls = 8 average_bytes = 8273 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 154, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 25, ] ] max_sm_queue_count = 6 proxy seq 58 2026-06-22 Mon 05:33:32:628.530 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:33:32:628.550 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 44 2026-06-22 Mon 05:33:32:628.560 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:33:32:628.594 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.121306 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 3 2026-06-22 Mon 05:33:32:628.605 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:33:32:628.613 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:33:32:628.653 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: Established 2026-06-22 Mon 05:33:57:689.645 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200368133696, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-22 Mon 05:34:57:690.377 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199606609472, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-22 Mon 05:35:57:692.122 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-06-22 Mon 05:36:57:693.135 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140199472391744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-06-22 Mon 05:37:00:216.735 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Processing scm::EvStop in state Established 2026-06-22 Mon 05:37:00:216.871 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Idle 2026-06-22 Mon 05:37:00:216.991 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-22 Mon 05:37:00:217.006 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-22 Mon 05:37:00:217.183 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: SANDESH: Send FAILED: 1782106620216972 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1782106192549585 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 3051 calls = 6 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 379188 calls = 43 average_bytes = 8818 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 154, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 90, _iter59->first = sent _iter59->second = 50, ] ] ] 2026-06-22 Mon 05:37:00:217.217 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 140200372332096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:37:21:839.412 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: http-server-port 8091 2026-06-22 Mon 05:37:21:839.485 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Max-tasks 16 2026-06-22 Mon 05:37:21:839.492 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Max-slice 100 2026-06-22 Mon 05:37:21:839.497 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Collectors 10.0.0.38:8086 2026-06-22 Mon 05:37:21:839.501 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Collectors 10.0.0.50:8086 2026-06-22 Mon 05:37:21:839.505 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Collectors 10.0.0.33:8086 2026-06-22 Mon 05:37:21:839.523 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: ROLE : Generator 2026-06-22 Mon 05:37:21:839.528 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-22 Mon 05:37:21:839.532 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-6019-3. 2026-06-22 Mon 05:37:21:839.537 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-22 Mon 05:37:21:839.541 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-22 Mon 05:37:21:839.545 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-22 Mon 05:37:21:839.550 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-22 Mon 05:37:21:863.226 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: HTTP Introspect Init 2026-06-22 Mon 05:37:21:863.362 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-06-22 Mon 05:37:21:863.456 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Sandesh Http Server Port: 8091 2026-06-22 Mon 05:37:21:863.519 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-22 Mon 05:37:21:863.529 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-22 Mon 05:37:21:863.922 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Idle 2026-06-22 Mon 05:37:21:864.067 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: No Client: 1782106641864014 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106641863939 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:37:21:864.228 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-22 Mon 05:37:21:864.243 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-22 Mon 05:37:21:864.255 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-22 Mon 05:37:21:891.931 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvStart in state Idle 2026-06-22 Mon 05:37:21:892.088 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Disconnect 2026-06-22 Mon 05:37:21:892.180 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-22 Mon 05:37:21:892.454 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-22 Mon 05:37:21:892.516 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:37:21:892.534 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SANDESH: Send FAILED: 1782106641892075 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:37:21:892.549 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:37:21:892.556 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:37:21:892.623 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SANDESH: Send FAILED: 1782106641892168 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106641863939 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:37:21:892.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:37:21:892.674 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:37:21:892.716 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SANDESH: Send FAILED: 1782106641892234 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106641863939 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:37:21:892.728 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:37:21:892.734 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:37:21:892.748 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SANDESH: Send FAILED: 1782106641892254 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:37:21:892.755 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:37:21:892.763 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:37:21:892.801 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SANDESH: Send FAILED: 1782106641892503 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1782106641863939 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:37:21:892.812 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-22 Mon 05:37:21:996.587 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-22 Mon 05:37:22:010.549 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-22 Mon 05:37:22:010.565 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x27ff540 2026-06-22 Mon 05:37:22:010.736 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.751 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.777 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x27fa580 2026-06-22 Mon 05:37:22:010.782 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.788 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.797 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x27ffeb0 2026-06-22 Mon 05:37:22:010.801 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.807 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.816 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x28061b0 2026-06-22 Mon 05:37:22:010.820 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.825 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.834 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x28024f0 2026-06-22 Mon 05:37:22:010.838 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.843 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.851 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2801830 2026-06-22 Mon 05:37:22:010.855 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.860 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.868 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2803cf0 2026-06-22 Mon 05:37:22:010.872 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.878 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.886 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2800b70 2026-06-22 Mon 05:37:22:010.890 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.895 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.904 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x28030b0 2026-06-22 Mon 05:37:22:010.908 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.913 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.923 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2804930 2026-06-22 Mon 05:37:22:010.927 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.933 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.940 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2805570 2026-06-22 Mon 05:37:22:010.944 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.950 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.958 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2806df0 2026-06-22 Mon 05:37:22:010.961 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.967 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.974 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2807a30 2026-06-22 Mon 05:37:22:010.978 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:010.984 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:010.992 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x2808670 2026-06-22 Mon 05:37:22:010.996 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:011.001 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:011.009 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: RAC_Connect status: 0 0x28092b0 2026-06-22 Mon 05:37:22:011.013 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: Connected to REDIS... 2026-06-22 Mon 05:37:22:011.018 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-22 Mon 05:37:22:012.583 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:52338::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-06-22 Mon 05:37:22:012.624 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: 10.0.0.33:52338::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-22 Mon 05:37:22:012.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: 10.0.0.33:52338::10.0.0.50:8086(-1):EvTcpConnected 2026-06-22 Mon 05:37:22:012.939 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:012.971 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:012.991 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.001 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.015 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.025 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.068 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.146 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.158 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.192 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.200 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.210 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.216 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.231 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.241 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.257 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.263 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.273 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.279 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.293 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.300 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.309 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.315 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.324 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.330 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.339 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.345 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.355 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.361 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.371 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.377 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.386 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.392 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.402 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-22 Mon 05:37:22:013.408 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-22 Mon 05:37:22:013.422 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.442 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.455 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.464 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.472 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.480 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.488 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.499 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.507 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.515 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-22 Mon 05:37:22:013.585 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642010462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-22 Mon 05:37:22:013.594 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-22 Mon 05:37:22:013.603 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-22 Mon 05:37:22:013.610 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-22 Mon 05:37:22:013.639 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit 2026-06-22 Mon 05:37:22:013.658 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-6019-3.:contrail-query-engine:0:Database count 1 2026-06-22 Mon 05:37:22:013.749 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.760 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.781 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642013060 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-22 Mon 05:37:22:013.787 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.793 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.798 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.811 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642013139 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:37:22:013.820 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.827 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.835 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.850 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642013437 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:37:22:013.856 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.870 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.877 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.887 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642013452 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:37:22:013.891 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.897 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.902 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.911 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642013633 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-22 Mon 05:37:22:013.917 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-22 Mon 05:37:22:013.922 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.927 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-22 Mon 05:37:22:013.946 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-22 Mon 05:37:22:013.951 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-22 Mon 05:37:22:014.086 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SANDESH: Send FAILED: 1782106642013734 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782106641863939 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 63, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-22 Mon 05:37:22:014.105 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-22 Mon 05:37:22:018.823 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: OnMessage control in state: ClientInit 2026-06-22 Mon 05:37:22:018.893 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-22 Mon 05:37:22:018.903 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-22 Mon 05:37:22:018.925 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: Received Ctrl Message with size 0 2026-06-22 Mon 05:37:22:019.059 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1782106641863939 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 63, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2026-06-22 Mon 05:37:22:019.093 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:37:22:019.119 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-6019-3. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 8 2026-06-22 Mon 05:37:22:019.132 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-22 Mon 05:37:22:019.141 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:37:22:019.148 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-22 Mon 05:37:22:019.188 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: Established 2026-06-22 Mon 05:38:21:864.768 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712007611968, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-22 Mon 05:39:21:865.630 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-22 Mon 05:40:21:866.353 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-22 Mon 05:41:21:867.413 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712007611968, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-22 Mon 05:42:21:868.173 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712035493440, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-22 Mon 05:43:21:868.878 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-06-22 Mon 05:44:21:869.599 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-06-22 Mon 05:45:21:870.257 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712039691840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-06-22 Mon 05:46:21:871.362 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712043890240, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-06-22 Mon 05:47:21:872.314 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712035493440, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-06-22 Mon 05:48:21:872.908 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-06-22 Mon 05:49:21:873.580 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712048088640, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-06-22 Mon 05:50:21:874.348 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711456732736, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-06-22 Mon 05:51:19:397.112 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.33:43974 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-22 Mon 05:51:21:875.154 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-06-22 Mon 05:51:37:329.768 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.33:43918 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-22 Mon 05:52:21:876.120 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712039691840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 15 without seqno , total = 0 2026-06-22 Mon 05:52:48:639.334 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.38:41686 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-22 Mon 05:53:16:919.479 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-06-22 Mon 05:53:16:920.158 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID aa2978f4-6dfe-11f1-bbd3-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-06-22 Mon 05:53:16:920.230 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QEQueryLog: aa2978f4-6dfe-11f1-bbd3-00000a000026 :0:json query is: { end_time: 1782107596907568 , enqueue_time: 1782107596917878 , query_metadata: {"enqueue_time": 1782107596917744} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1782107572388132 , table: "ObjectRoutingInstance" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-87061349:ctest-vn22-35009221:ctest-vn22-35009221", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-22 Mon 05:53:16:920.422 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-06-22 Mon 05:53:16:920.436 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-06-22 Mon 05:53:16:920.481 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for aa2978f4-6dfe-11f1-bbd3-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-06-22 Mon 05:53:16:920.574 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QEQueryLog: aa2978f4-6dfe-11f1-bbd3-00000a000026 :0:json query is: { end_time: 1782107596907568 , enqueue_time: 1782107596917878 , query_metadata: {"enqueue_time": 1782107596917744} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1782107572388132 , table: "ObjectRoutingInstance" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-87061349:ctest-vn22-35009221:ctest-vn22-35009221", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-22 Mon 05:53:17:002.241 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:002.309 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:002.318 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:002.326 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:002.333 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:002.361 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:002.368 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:002.375 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:002.383 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:002.390 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:002.460 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: [SYS_INFO]: QEQueryLog: aa2978f4-6dfe-11f1-bbd3-00000a000026 :0:json query is: { end_time: 1782107596907568 , enqueue_time: 1782107596917878 , query_metadata: {"enqueue_time": 1782107596917744} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1782107572388132 , table: "ObjectRoutingInstance" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-87061349:ctest-vn22-35009221:ctest-vn22-35009221", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-22 Mon 05:53:17:004.147 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.184 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.204 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.211 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.217 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.222 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.240 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.258 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.266 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.274 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.699 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QEQueryLog: aa2978f4-6dfe-11f1-bbd3-00000a000026 ObjectRoutingInstance :2:Column ObjectTypeName1 Table ObjectRoutingInstance object_id default-domain:ctest-AnalyticsBasicTestSanity-87061349:ctest-vn22-35009221:ctest-vn22-35009221 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-22 Mon 05:53:17:004.790 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.798 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.804 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.810 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.815 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.821 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.826 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.832 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:004.837 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-22 Mon 05:53:17:004.849 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: aa2978f4-6dfe-11f1-bbd3-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-22 Mon 05:53:17:005.075 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-06-22 Mon 05:53:17:005.126 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-06-22 Mon 05:53:17:005.169 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711452534336, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":1, "count":1} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-06-22 Mon 05:53:17:007.530 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: [SYS_INFO]: QELog: Finished: QID aa2978f4-6dfe-11f1-bbd3-00000a000026 Table ObjectRoutingInstance Time(ms) 87 RedisTime(ms) 0 MergeTime(ms) 0 Rows 1 EnQ-delay2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-06-22 Mon 05:53:17:007.565 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712007611968, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-6019-3. table = ObjectRoutingInstance query_stats= [ qid = aa2978f4-6dfe-11f1-bbd3-00000a000026 where = [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-87061349:ctest-vn22-35009221:ctest-vn22-35009221", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 83,0, 81,0, 82,0, chunk_select_time = 0,0, 0,0, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 87 rows = 1 enq_delay = 2 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-06-22 Mon 05:53:17:007.616 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712003413568, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-06-22 Mon 05:53:21:877.051 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711448335936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 16 without seqno , total = 0 2026-06-22 Mon 05:54:21:877.841 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712056485440, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 17 without seqno , total = 0 2026-06-22 Mon 05:55:21:878.972 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711460931136, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 18 without seqno , total = 0 2026-06-22 Mon 05:56:21:879.633 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 19 without seqno , total = 0 2026-06-22 Mon 05:57:21:882.937 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712043890240, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 20 without seqno , total = 0 2026-06-22 Mon 05:58:21:883.576 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712056485440, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 21 without seqno , total = 0 2026-06-22 Mon 05:59:21:884.267 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712056485440, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 22 without seqno , total = 0 2026-06-22 Mon 06:00:21:885.740 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712011810368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 23 without seqno , total = 0 2026-06-22 Mon 06:01:21:887.115 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712031295040, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 24 without seqno , total = 0 2026-06-22 Mon 06:02:21:887.808 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712016008768, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 25 without seqno , total = 0 2026-06-22 Mon 06:03:21:888.536 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 26 without seqno , total = 0 2026-06-22 Mon 06:04:21:889.269 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712043890240, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 27 without seqno , total = 0 2026-06-22 Mon 06:05:21:889.885 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712039691840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 28 without seqno , total = 0 2026-06-22 Mon 06:05:46:173.676 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.38:42258 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-22 Mon 06:06:21:890.833 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712060683840, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 29 without seqno , total = 0 2026-06-22 Mon 06:07:18:312.479 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139712067823744, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8091::10.0.0.33:41716 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-22 Mon 06:07:21:891.629 UTC cn-jenkins-deploy-platform-ansible-os-6019-3 [Thread 139711448335936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 30 without seqno , total = 0