2026-03-26 Thu 05:28:15:059.812 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: http-server-port 8091 2026-03-26 Thu 05:28:15:059.923 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Max-tasks 16 2026-03-26 Thu 05:28:15:059.933 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Max-slice 100 2026-03-26 Thu 05:28:15:059.940 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Collectors 10.0.0.38:8086 2026-03-26 Thu 05:28:15:059.945 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Collectors 10.0.0.242:8086 2026-03-26 Thu 05:28:15:059.951 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Collectors 10.0.0.33:8086 2026-03-26 Thu 05:28:15:059.974 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: ROLE : Generator 2026-03-26 Thu 05:28:15:059.980 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-03-26 Thu 05:28:15:059.987 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5546-1. 2026-03-26 Thu 05:28:15:059.993 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: NODE TYPE : Database 2026-03-26 Thu 05:28:15:059.999 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: INSTANCE ID : 0 2026-03-26 Thu 05:28:15:060.005 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-03-26 Thu 05:28:15:060.013 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-03-26 Thu 05:28:15:060.687 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: HTTP Introspect Init 2026-03-26 Thu 05:28:15:061.110 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-03-26 Thu 05:28:15:061.219 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Sandesh Http Server Port: 8091 2026-03-26 Thu 05:28:15:061.278 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-03-26 Thu 05:28:15:061.291 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-03-26 Thu 05:28:15:066.334 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Idle 2026-03-26 Thu 05:28:15:067.622 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: No Client: 1774502895066463 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:28:15:067.941 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-03-26 Thu 05:28:15:067.968 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-03-26 Thu 05:28:15:067.991 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.242:9042 10.0.0.33:9042 2026-03-26 Thu 05:28:15:068.059 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvStart in state Idle 2026-03-26 Thu 05:28:15:068.146 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Disconnect 2026-03-26 Thu 05:28:15:068.227 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-03-26 Thu 05:28:15:068.473 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Connect : Start Connect timer 10.0.0.242:8086 2026-03-26 Thu 05:28:15:068.590 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:28:15:068.624 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774502895068136 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:28:15:068.635 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:28:15:068.646 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:28:15:068.737 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774502895068216 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:28:15:068.756 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:28:15:068.767 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:28:15:068.848 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774502895068282 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:28:15:068.862 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:28:15:068.873 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:28:15:068.890 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774502895068307 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:28:15:068.897 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:28:15:068.906 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:28:15:069.094 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774502895068571 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:28:15:069.117 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:28:15:090.778 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-26 Thu 05:28:15:090.816 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:15:093.577 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:28:15:093.647 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774502895090860 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:28:15:093.659 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:28:15:098.197 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 0 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:20:101.338 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-26 Thu 05:28:20:101.402 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:20:101.455 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 1 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:25:103.806 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-26 Thu 05:28:25:103.853 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:25:103.881 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 2 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:30:106.116 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-26 Thu 05:28:30:106.175 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:30:106.203 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 3 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:35:175.047 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-26 Thu 05:28:35:175.096 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:35:175.121 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 4 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:40:246.465 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-26 Thu 05:28:40:246.529 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:40:246.556 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 5 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:45:279.415 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-26 Thu 05:28:45:279.469 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:45:279.510 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 6 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:50:344.804 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-26 Thu 05:28:50:344.856 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-26 Thu 05:28:50:344.883 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 7 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:28:55:390.923 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-03-26 Thu 05:28:55:390.990 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 8 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-26 Thu 05:29:00:416.359 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-03-26 Thu 05:29:05:421.702 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453013878528, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:29:05:421.789 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453013878528, Pid 1]: SANDESH: Send FAILED: 1774502945421599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:29:05:421.803 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453013878528, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:05:421.750 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-26 Thu 05:29:05:422.042 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1):EvTcpConnectFail 2026-03-26 Thu 05:29:05:422.079 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14045f0 2026-03-26 Thu 05:29:05:422.121 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:422.099 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-26 Thu 05:29:05:422.176 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Connect : EvTcpConnectFail 2026-03-26 Thu 05:29:05:422.356 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:29:05:422.408 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Idle 2026-03-26 Thu 05:29:05:422.494 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:422.619 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SANDESH: Send FAILED: 1774502945422293 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 29, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:29:05:422.644 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:29:05:422.659 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:422.682 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SANDESH: Send FAILED: 1774502945422328 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:29:05:422.690 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:05:422.700 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:29:05:422.746 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:422.864 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SANDESH: Send FAILED: 1774502945422481 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 30, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:29:05:422.880 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:29:05:422.141 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.106 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1404ca0 2026-03-26 Thu 05:29:05:423.113 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.126 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.138 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x13fe1d0 2026-03-26 Thu 05:29:05:423.144 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.155 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.165 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1405340 2026-03-26 Thu 05:29:05:423.171 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.181 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.198 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1405870 2026-03-26 Thu 05:29:05:423.204 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.213 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.224 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14061a0 2026-03-26 Thu 05:29:05:423.230 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.240 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.251 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1407420 2026-03-26 Thu 05:29:05:423.257 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.266 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.276 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1408fe0 2026-03-26 Thu 05:29:05:423.282 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.297 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.307 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1406ae0 2026-03-26 Thu 05:29:05:423.313 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.323 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.333 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1409920 2026-03-26 Thu 05:29:05:423.339 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.348 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.358 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140a260 2026-03-26 Thu 05:29:05:423.364 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.372 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.382 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1407d60 2026-03-26 Thu 05:29:05:423.388 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.397 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.410 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140aba0 2026-03-26 Thu 05:29:05:423.417 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.426 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.435 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140b4e0 2026-03-26 Thu 05:29:05:423.441 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.451 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.479 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14086a0 2026-03-26 Thu 05:29:05:423.486 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:29:05:423.495 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:29:05:423.662 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:423.722 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774502945423551 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774502945422406 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774502945422745 ] max_sm_queue_count = 5 ] 2026-03-26 Thu 05:29:05:423.732 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:29:05:423.743 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:423.895 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774502945423638 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 44, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:29:05:423.918 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:29:05:424.211 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.235 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.252 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.264 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.279 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.289 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.303 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.314 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.327 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.337 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.400 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.455 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.497 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.520 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.538 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.405 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:424.653 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774502945424382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:29:05:424.617 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.692 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.708 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.719 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.734 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.745 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.758 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.769 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.783 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.793 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.807 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.818 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.832 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:424.842 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:424.873 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.917 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.936 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.956 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.973 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.997 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:425.022 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:425.039 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:425.051 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:425.065 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:425.076 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:425.090 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:29:05:425.101 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:29:05:425.134 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:425.150 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:425.165 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:29:05:424.673 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:05:425.399 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:425.422 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774502945424445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.242:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:29:05:425.434 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:05:425.444 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:425.474 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774502945424896 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.242:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:29:05:425.482 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:05:425.491 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:29:05:425.514 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774502945425016 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:29:05:425.522 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:06:425.432 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:29:06:425.808 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-26 Thu 05:29:06:425.977 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:29:06:426.036 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774502946425551 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:29:06:426.050 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:29:06:426.065 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:29:06:426.292 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774502946425953 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 89, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:29:06:426.317 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:29:06:428.121 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:49770::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-26 Thu 05:29:06:428.161 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:49770::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:29:06:428.171 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:49770::10.0.0.38:8086(-1):EvTcpConnected 2026-03-26 Thu 05:29:06:428.200 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:29:06:428.224 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:29:06:428.267 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit 2026-03-26 Thu 05:29:06:428.294 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 1 2026-03-26 Thu 05:29:06:428.401 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:29:06:428.413 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:29:06:428.447 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774502946428258 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:29:06:428.471 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:29:06:428.482 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:29:06:428.490 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:29:06:428.504 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:29:06:428.523 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:29:06:428.652 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774502946428387 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:29:06:428.671 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:29:06:437.498 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: OnMessage control in state: ClientInit 2026-03-26 Thu 05:29:06:437.567 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-26 Thu 05:29:06:437.583 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-26 Thu 05:29:06:437.622 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Received Ctrl Message with size 0 2026-03-26 Thu 05:29:06:437.796 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774502945422406 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774502945422745 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 10 2026-03-26 Thu 05:29:06:437.831 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:29:06:437.862 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 11 2026-03-26 Thu 05:29:06:437.876 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:29:06:437.884 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:29:06:437.893 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:29:06:437.931 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Established 2026-03-26 Thu 05:29:16:120.976 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: OnMessage control in state: Established 2026-03-26 Thu 05:29:16:121.052 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-03-26 Thu 05:29:16:121.063 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Established : EvSandeshMessageRecv 2026-03-26 Thu 05:29:16:121.112 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Received Ctrl Message with size 3 2026-03-26 Thu 05:29:16:121.127 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 11, total = 0 2026-03-26 Thu 05:29:16:121.138 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 13, total = 0 2026-03-26 Thu 05:29:16:121.151 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 1, total = 0 2026-03-26 Thu 05:29:16:121.159 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:29:35:424.074 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-03-26 Thu 05:30:35:424.691 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-03-26 Thu 05:31:35:425.390 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-03-26 Thu 05:32:35:425.975 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-03-26 Thu 05:33:35:428.777 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-03-26 Thu 05:33:50:731.007 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:49308 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2026-03-26 Thu 05:34:05:865.715 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:51576 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2026-03-26 Thu 05:34:35:430.403 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-03-26 Thu 05:35:35:433.351 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453013878528, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-03-26 Thu 05:36:22:223.013 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x140b4e0 2026-03-26 Thu 05:36:22:223.078 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:223.129 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x140aba0 2026-03-26 Thu 05:36:22:223.143 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:223.167 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x140a260 2026-03-26 Thu 05:36:22:223.178 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:223.207 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1409920 2026-03-26 Thu 05:36:22:223.219 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:223.246 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-26 Thu 05:36:22:223.254 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1408fe0 2026-03-26 Thu 05:36:22:223.264 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:223.612 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:36:22:223.651 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:36:22:223.672 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:36:22:223.686 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:36:22:223.712 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:36:22:527.495 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x14086a0 2026-03-26 Thu 05:36:22:527.559 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:527.600 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1407d60 2026-03-26 Thu 05:36:22:527.612 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:527.927 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1407420 2026-03-26 Thu 05:36:22:527.955 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:527.983 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1406ae0 2026-03-26 Thu 05:36:22:527.995 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:528.021 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-26 Thu 05:36:22:528.029 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x14061a0 2026-03-26 Thu 05:36:22:528.039 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:36:22:528.233 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:36:22:528.259 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:36:22:528.661 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:36:22:528.709 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:36:22:528.728 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:36:27:223.704 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:36:27:223.868 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:36:27:223.913 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:36:27:223.949 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:36:27:223.983 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:36:27:224.053 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:36:27:224.087 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:36:27:224.102 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:36:27:224.116 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:36:27:224.132 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:36:27:528.299 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:36:27:528.459 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:36:27:528.712 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:36:27:528.768 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:36:27:528.801 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:36:27:528.867 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:36:27:528.901 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:36:27:529.028 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:36:27:529.113 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:36:27:529.134 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:36:32:224.155 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:36:32:224.324 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:36:32:224.373 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:36:32:224.414 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:36:32:224.447 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:36:32:224.495 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:36:32:224.523 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:36:32:224.540 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:36:32:224.559 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:36:32:224.579 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:36:32:528.937 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:36:32:529.132 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:36:32:529.176 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:36:32:529.214 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:36:32:529.249 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:36:32:529.458 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:36:32:529.490 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:36:32:529.507 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:36:32:529.521 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:36:32:529.535 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:36:35:434.818 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-03-26 Thu 05:36:37:224.598 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:36:37:224.804 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:36:37:224.858 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:36:37:224.917 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140b4e0 2026-03-26 Thu 05:36:37:224.929 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:37:224.974 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:37:224.994 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140aba0 2026-03-26 Thu 05:36:37:225.004 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:37:225.021 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:37:225.034 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:36:37:225.084 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:36:37:225.156 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140a260 2026-03-26 Thu 05:36:37:225.166 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:37:225.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:37:225.255 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1409920 2026-03-26 Thu 05:36:37:225.265 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:37:225.281 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:37:225.295 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1408fe0 2026-03-26 Thu 05:36:37:225.304 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:37:225.319 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:37:225.549 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:37:225.582 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:37:225.601 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:37:225.612 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:37:225.640 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:37:225.710 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:37:225.729 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:37:225.739 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:37:225.753 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:37:225.764 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:37:225.888 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:37:225.904 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:37:225.926 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:37:225.954 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:37:225.988 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:37:529.563 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:36:37:529.707 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:36:37:529.769 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:36:37:529.832 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:36:37:529.890 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:36:37:529.998 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:36:37:530.056 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:36:37:530.081 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:36:37:530.186 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:36:37:530.218 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:36:42:531.798 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:36:42:532.035 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:36:42:532.076 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:36:42:532.112 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:36:42:532.146 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:36:42:533.026 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1407d60 2026-03-26 Thu 05:36:42:533.060 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:42:533.091 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:42:533.106 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1407420 2026-03-26 Thu 05:36:42:533.113 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:42:533.123 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:42:533.133 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1406ae0 2026-03-26 Thu 05:36:42:533.149 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:42:533.158 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:42:533.168 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14061a0 2026-03-26 Thu 05:36:42:533.174 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:42:533.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:42:533.193 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14086a0 2026-03-26 Thu 05:36:42:533.199 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:36:42:533.208 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:36:42:534.857 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:42:534.882 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:42:534.899 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:42:534.910 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:42:534.925 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:42:534.935 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:42:534.949 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:42:534.959 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:42:534.973 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:36:42:534.983 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:36:42:535.009 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:42:544.854 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:42:544.912 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:42:544.928 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:36:42:544.944 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:37:35:440.644 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452966782720, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-03-26 Thu 05:38:35:445.306 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-03-26 Thu 05:39:35:446.196 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452992890624, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-03-26 Thu 05:39:43:499.978 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-26 Thu 05:39:43:500.029 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1408fe0 2026-03-26 Thu 05:39:43:500.054 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:500.109 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1409920 2026-03-26 Thu 05:39:43:500.122 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:500.161 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x140a260 2026-03-26 Thu 05:39:43:500.174 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:500.203 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x140aba0 2026-03-26 Thu 05:39:43:500.214 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:500.236 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x140b4e0 2026-03-26 Thu 05:39:43:500.247 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:503.080 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:39:43:503.139 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:39:43:503.166 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:39:43:503.185 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:39:43:503.201 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:39:43:617.557 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1405870 2026-03-26 Thu 05:39:43:617.622 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:617.792 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1405340 2026-03-26 Thu 05:39:43:617.811 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:617.845 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x13fe1d0 2026-03-26 Thu 05:39:43:617.857 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:617.884 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1404ca0 2026-03-26 Thu 05:39:43:617.895 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:617.926 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-26 Thu 05:39:43:617.935 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x14045f0 2026-03-26 Thu 05:39:43:617.944 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:623.835 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:39:43:623.928 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:39:43:623.968 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:39:43:623.995 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:39:43:624.027 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:39:43:624.047 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x14086a0 2026-03-26 Thu 05:39:43:624.078 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:624.114 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-26 Thu 05:39:43:624.130 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x14061a0 2026-03-26 Thu 05:39:43:624.141 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:624.165 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1407420 2026-03-26 Thu 05:39:43:624.176 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:624.198 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1407d60 2026-03-26 Thu 05:39:43:624.208 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:624.229 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Disconnect status: -1 0x1406ae0 2026-03-26 Thu 05:39:43:624.239 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-26 Thu 05:39:43:632.992 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:49770::10.0.0.38:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:39:43:633.046 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:49770::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-26 Thu 05:39:43:633.057 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:49770::10.0.0.38:8086(-1):EvTcpClose 2026-03-26 Thu 05:39:43:642.376 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvTcpClose in state Established 2026-03-26 Thu 05:39:43:645.714 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Established : EvTcpClose 2026-03-26 Thu 05:39:43:646.013 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-26 Thu 05:39:43:646.295 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:43:646.477 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SANDESH: Send FAILED: 1774503583645823 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 1 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = cn-jenkins-deploy-platform-ansible-os-5546-1. collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 562927 calls = 65 average_bytes = 8660 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 163, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 66, ] ] ] 2026-03-26 Thu 05:39:43:646.502 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:43:646.514 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:43:646.542 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SANDESH: Send FAILED: 1774503583645858 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:43:646.551 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:43:646.561 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-03-26 Thu 05:39:43:646.572 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:43:646.729 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SANDESH: Send FAILED: 1774503583646275 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 562927 calls = 65 average_bytes = 8660 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 163, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 66, ] ] ] 2026-03-26 Thu 05:39:43:646.747 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:43:642.432 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:39:43:646.988 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:39:43:647.007 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:39:43:647.025 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:39:43:647.042 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:39:43:649.838 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:54080::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-26 Thu 05:39:43:649.864 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:39:43:649.873 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1):EvTcpConnected 2026-03-26 Thu 05:39:43:652.290 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:39:43:652.306 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:39:43:652.347 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit 2026-03-26 Thu 05:39:43:652.366 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 2 2026-03-26 Thu 05:39:43:652.458 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:43:652.477 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:39:43:652.504 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774503583652338 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:43:652.513 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:39:43:652.523 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:43:652.531 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:39:43:652.544 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:43:652.551 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:39:43:652.715 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774503583652447 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 562927 calls = 65 average_bytes = 8660 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 164, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 66, ] ] ] 2026-03-26 Thu 05:39:43:652.734 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:39:43:740.485 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-26 Thu 05:39:43:740.524 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1):EvTcpClose 2026-03-26 Thu 05:39:43:743.020 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-26 Thu 05:39:43:743.046 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvTcpClose 2026-03-26 Thu 05:39:43:743.251 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Return to idle with 1000 2026-03-26 Thu 05:39:43:743.263 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Idle 2026-03-26 Thu 05:39:43:743.333 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:43:743.500 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503583743152 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 564500 calls = 66 average_bytes = 8553 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 164, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 67, ] ] ] 2026-03-26 Thu 05:39:43:743.525 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:43:743.538 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:43:743.564 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503583743213 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:43:743.572 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:43:743.582 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:39:43:743.593 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:43:743.755 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503583743322 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 2 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 564500 calls = 66 average_bytes = 8553 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 164, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 67, ] ] ] 2026-03-26 Thu 05:39:43:743.772 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:44:745.583 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:39:44:745.833 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: Connect : Start Connect timer 10.0.0.242:8086 2026-03-26 Thu 05:39:44:745.968 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:44:746.003 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: SANDESH: Send FAILED: 1774503584745681 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:44:746.014 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:44:746.025 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:44:746.235 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: SANDESH: Send FAILED: 1774503584745952 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 564500 calls = 66 average_bytes = 8553 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 164, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 67, ] ] ] 2026-03-26 Thu 05:39:44:746.264 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:44:753.126 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:35312::10.0.0.242:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-26 Thu 05:39:44:753.174 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:35312::10.0.0.242:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:39:44:753.188 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:35312::10.0.0.242:8086(-1):EvTcpConnected 2026-03-26 Thu 05:39:44:753.254 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:39:44:753.284 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:39:44:753.332 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit 2026-03-26 Thu 05:39:44:753.357 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 3 2026-03-26 Thu 05:39:44:753.457 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:44:753.468 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:39:44:753.495 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503584753323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:44:753.504 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:39:44:753.514 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:44:753.522 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:39:44:753.534 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:44:753.552 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:39:44:754.828 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503584753446 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 564500 calls = 66 average_bytes = 8553 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 67, ] ] ] 2026-03-26 Thu 05:39:44:754.882 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:39:44:756.700 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:35312::10.0.0.242:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-26 Thu 05:39:44:756.721 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:35312::10.0.0.242:8086(-1):EvTcpClose 2026-03-26 Thu 05:39:44:757.683 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-26 Thu 05:39:44:757.702 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: ClientInit : EvTcpClose 2026-03-26 Thu 05:39:44:757.855 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Return to idle with 1000 2026-03-26 Thu 05:39:44:757.866 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Idle 2026-03-26 Thu 05:39:44:757.937 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:44:758.135 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: SANDESH: Send FAILED: 1774503584757796 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 566073 calls = 67 average_bytes = 8448 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 68, ] ] ] 2026-03-26 Thu 05:39:44:758.160 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:44:758.172 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:44:758.197 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: SANDESH: Send FAILED: 1774503584757829 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:44:758.205 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:44:758.215 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:39:44:758.225 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:44:758.379 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: SANDESH: Send FAILED: 1774503584757925 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 3 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 566073 calls = 67 average_bytes = 8448 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 68, ] ] ] 2026-03-26 Thu 05:39:44:758.395 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:45:760.497 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:39:45:760.926 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-26 Thu 05:39:45:761.098 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:45:761.135 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503585760593 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:45:761.159 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:45:761.174 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:45:761.371 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503585761075 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 3 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 566073 calls = 67 average_bytes = 8448 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 68, ] ] ] 2026-03-26 Thu 05:39:45:761.391 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:45:766.354 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:59214::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-26 Thu 05:39:45:766.412 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:59214::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:39:45:766.426 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:59214::10.0.0.38:8086(-1):EvTcpConnected 2026-03-26 Thu 05:39:45:767.006 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:39:45:767.050 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:39:45:767.124 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: ClientInit 2026-03-26 Thu 05:39:45:767.160 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 4 2026-03-26 Thu 05:39:45:767.276 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:45:767.288 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:39:45:767.316 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: SANDESH: Send FAILED: 1774503585767113 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:45:767.326 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:39:45:767.336 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:45:767.344 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:39:45:767.360 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:45:767.367 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:39:45:767.526 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: SANDESH: Send FAILED: 1774503585767261 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 566073 calls = 67 average_bytes = 8448 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 68, ] ] ] 2026-03-26 Thu 05:39:45:767.550 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:39:48:503.203 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:39:48:503.394 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:39:48:503.454 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:39:48:503.496 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:39:48:503.547 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:39:48:503.621 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:39:48:503.658 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:39:48:503.673 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:39:48:503.696 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:39:48:503.719 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:39:48:623.923 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:39:48:624.145 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:39:48:624.179 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:39:48:624.222 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:39:48:624.260 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:39:48:624.293 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:39:48:624.334 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:39:48:624.352 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:39:48:624.367 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:39:48:624.395 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:39:48:646.986 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:39:48:647.176 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:39:48:647.247 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:39:48:647.293 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:39:48:647.327 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:39:48:647.576 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:39:48:647.652 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:39:48:647.672 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:39:48:647.689 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:39:48:647.706 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:39:49:767.985 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: 10.0.0.38:8086 EvConnectTimerExpired in state ClientInit 2026-03-26 Thu 05:39:49:768.055 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-03-26 Thu 05:39:49:768.067 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvConnectTimerExpired 2026-03-26 Thu 05:39:49:768.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Return to idle with 1000 2026-03-26 Thu 05:39:49:768.271 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Idle 2026-03-26 Thu 05:39:49:768.342 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:49:768.521 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503589768157 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 567646 calls = 68 average_bytes = 8347 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 39, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-26 Thu 05:39:49:768.561 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:49:768.578 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:49:768.614 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503589768191 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:49:768.626 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:49:768.641 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:39:49:768.656 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:49:768.811 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503589768330 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 4 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 567646 calls = 68 average_bytes = 8347 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 39, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-26 Thu 05:39:49:768.829 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:50:769.032 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:39:50:769.280 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-26 Thu 05:39:50:769.408 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:50:769.440 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774503590769121 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:50:769.451 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:50:769.461 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:50:769.682 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: SANDESH: Send FAILED: 1774503590769391 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 4 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 567646 calls = 68 average_bytes = 8347 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-26 Thu 05:39:50:769.711 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452962584320, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:50:773.749 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:33268::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-26 Thu 05:39:50:773.790 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:33268::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:39:50:773.802 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:33268::10.0.0.33:8086(-1):EvTcpConnected 2026-03-26 Thu 05:39:50:774.598 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:39:50:774.635 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:39:50:774.694 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit 2026-03-26 Thu 05:39:50:774.719 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 5 2026-03-26 Thu 05:39:50:774.857 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:50:774.872 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:39:50:774.910 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503590774681 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:50:774.937 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:39:50:774.953 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:50:774.965 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:39:50:774.982 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:39:50:774.993 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:39:50:775.289 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503590774841 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 567646 calls = 68 average_bytes = 8347 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-26 Thu 05:39:50:775.321 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:39:53:503.704 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:39:53:503.900 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:39:53:503.972 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:39:53:504.011 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:39:53:504.049 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:39:53:504.104 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:39:53:504.135 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:39:53:504.150 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:39:53:504.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:39:53:504.198 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:39:53:624.205 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:39:53:624.393 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:39:53:624.424 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:39:53:624.491 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:39:53:624.536 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:39:53:624.554 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:39:53:624.566 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:39:53:624.599 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:39:53:624.641 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:39:53:624.666 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:39:53:647.679 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:39:53:647.844 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:39:53:647.896 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:39:53:647.934 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:39:53:647.967 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:39:53:648.157 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:39:53:648.204 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:39:53:648.220 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:39:53:648.236 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:39:53:648.251 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:39:55:403.377 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:33268::10.0.0.33:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2026-03-26 Thu 05:39:55:403.444 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:33268::10.0.0.33:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:39:55:403.468 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:33268::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-26 Thu 05:39:55:403.478 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:33268::10.0.0.33:8086(-1):EvTcpClose 2026-03-26 Thu 05:39:55:405.111 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-26 Thu 05:39:55:405.134 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: ClientInit : EvTcpClose 2026-03-26 Thu 05:39:55:405.301 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Return to idle with 1000 2026-03-26 Thu 05:39:55:405.313 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Idle 2026-03-26 Thu 05:39:55:405.382 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:55:405.560 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503595405230 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 46, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:39:55:405.592 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:55:405.604 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:55:405.630 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503595405270 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:55:405.639 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:55:405.648 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:39:55:405.659 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:55:405.804 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: SANDESH: Send FAILED: 1774503595405371 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 46, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:39:55:405.822 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453117794048, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:56:406.147 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:39:56:406.407 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Connect : Start Connect timer 10.0.0.242:8086 2026-03-26 Thu 05:39:56:406.527 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:56:406.562 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: SANDESH: Send FAILED: 1774503596406234 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:56:406.573 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:56:406.583 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:39:56:406.761 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: SANDESH: Send FAILED: 1774503596406509 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:39:56:406.787 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:56:408.911 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-26 Thu 05:39:56:408.923 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1):EvTcpConnectFail 2026-03-26 Thu 05:39:56:411.466 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-26 Thu 05:39:56:411.479 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Connect : EvTcpConnectFail 2026-03-26 Thu 05:39:56:411.601 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:39:56:411.636 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Idle 2026-03-26 Thu 05:39:56:411.704 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:56:411.896 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: SANDESH: Send FAILED: 1774503596411550 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:39:56:411.923 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:56:411.936 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:56:411.962 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: SANDESH: Send FAILED: 1774503596411578 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:39:56:411.970 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:39:56:411.980 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:39:56:411.991 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:39:56:412.138 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: SANDESH: Send FAILED: 1774503596411694 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:39:56:412.155 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453005485824, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:39:58:504.229 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:39:58:504.489 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:39:58:504.569 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:39:58:504.603 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:39:58:504.680 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:39:58:504.721 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:39:58:504.767 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:39:58:504.790 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:39:58:504.808 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:39:58:504.825 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:39:58:624.465 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:39:58:624.646 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:39:58:624.682 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:39:58:624.722 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:39:58:624.764 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:39:58:624.782 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:39:58:624.801 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:39:58:624.834 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:39:58:624.869 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:39:58:624.885 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:39:58:648.242 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:39:58:648.385 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:39:58:648.424 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:39:58:648.457 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:39:58:648.493 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:39:58:648.677 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:39:58:648.710 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:39:58:649.326 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:39:58:649.372 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:39:58:649.390 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:03:504.652 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:03:504.845 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:03:504.921 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:03:504.969 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:03:505.015 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:03:505.079 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:03:505.114 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:03:505.137 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:03:505.160 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:03:505.182 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:03:624.725 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:03:624.909 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:03:624.956 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:03:624.999 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:03:625.034 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:03:625.196 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:03:625.219 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:03:625.234 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:03:625.246 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:03:625.297 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:03:648.745 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:03:648.899 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:03:649.377 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:03:649.433 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:03:649.469 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:03:649.514 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:03:649.545 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:03:649.729 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:03:649.797 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:03:649.827 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:05:446.513 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:05:446.631 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: SANDESH: Send FAILED: 1774503605446369 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 124 dequeues = 124 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 152 dequeues = 152 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774503596411635 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774503596411989 ] max_sm_queue_count = 5 ] 2026-03-26 Thu 05:40:05:446.656 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:05:446.667 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:05:446.839 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: SANDESH: Send FAILED: 1774503605446487 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:05:446.856 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:08:505.182 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:08:505.403 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:08:505.501 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:08:505.558 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:08:505.666 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:08:505.728 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:08:505.776 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:08:505.809 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:08:505.831 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:08:505.853 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:08:625.073 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:08:625.339 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:08:625.379 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:08:625.420 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:08:625.456 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:08:625.489 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:08:625.531 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:08:625.548 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:08:625.564 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:08:625.579 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:08:649.630 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:08:649.858 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:08:649.911 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:08:649.946 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:08:649.981 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:08:650.200 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:08:650.273 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:08:650.314 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:08:650.332 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:08:650.395 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:10:445.171 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:10:445.280 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503610445128 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-03-26 Thu 05:40:10:445.293 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-03-26 Thu 05:40:12:411.772 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:40:12:412.004 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-26 Thu 05:40:12:412.111 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:40:12:412.144 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503612411859 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:40:12:412.155 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:40:12:412.166 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:40:12:412.344 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503612412098 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:12:412.373 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:12:412.795 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-26 Thu 05:40:12:412.829 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1):EvTcpConnectFail 2026-03-26 Thu 05:40:12:412.869 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-26 Thu 05:40:12:412.884 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Connect : EvTcpConnectFail 2026-03-26 Thu 05:40:12:413.018 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:40:12:413.054 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Idle 2026-03-26 Thu 05:40:12:413.120 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:12:413.279 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774503612412963 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:12:413.315 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:12:413.326 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:12:413.351 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774503612412997 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:40:12:413.359 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:40:12:413.369 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:40:12:413.393 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:12:413.540 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: SANDESH: Send FAILED: 1774503612413109 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:12:413.558 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452997089024, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:13:505.837 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:13:506.008 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:13:506.068 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:13:506.121 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:13:506.160 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:13:506.218 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:13:506.253 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:13:506.271 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:13:506.427 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:13:506.524 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:13:625.439 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:13:625.635 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:13:625.666 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:13:625.704 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:13:625.737 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:13:625.786 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:13:625.827 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:13:625.845 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:13:625.859 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:13:625.873 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:13:650.298 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:13:650.510 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:13:650.579 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:13:650.631 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:13:650.682 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:13:650.924 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:13:650.979 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:13:650.998 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:13:651.021 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:13:651.045 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:18:506.304 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:18:506.470 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:18:506.523 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:18:506.565 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:18:506.648 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:18:506.675 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:18:506.688 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:18:506.740 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:18:506.757 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:18:506.832 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:18:625.713 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:18:625.920 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:18:625.952 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:18:626.003 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:18:626.047 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:18:626.115 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:18:626.173 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:18:626.196 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:18:626.218 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:18:626.237 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:18:651.010 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:18:651.163 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:18:651.206 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:18:651.238 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:18:651.269 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:18:651.611 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:18:651.650 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:18:651.668 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:18:651.682 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:18:651.698 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:23:506.713 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:23:506.894 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:23:506.941 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:23:506.976 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:23:507.009 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:23:507.148 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:23:507.182 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:23:507.211 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:23:507.226 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:23:507.240 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:23:626.053 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:23:626.283 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:23:626.317 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:23:626.368 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:23:626.410 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:23:626.445 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:23:626.484 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:23:626.503 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:23:626.518 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:23:626.532 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:23:651.691 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:23:651.828 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:23:651.882 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:23:651.917 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:23:651.950 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:23:652.172 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:23:652.206 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:23:652.231 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:23:652.304 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:23:652.322 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:28:507.230 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:28:507.410 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:28:507.538 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:28:507.594 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:28:507.641 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:28:507.703 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:28:507.738 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:28:507.753 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:28:507.772 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:28:507.794 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:28:626.395 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:28:626.631 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:28:626.678 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:28:626.727 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:28:626.776 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:28:626.832 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:28:626.875 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:28:626.895 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:28:626.911 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:28:626.925 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:28:652.327 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:28:652.539 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:28:652.597 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:28:652.652 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:28:652.686 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:28:652.933 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:28:652.971 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:28:652.988 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:28:653.018 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:28:653.034 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:33:507.782 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:33:507.930 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:33:507.996 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:33:508.041 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:33:508.129 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:33:508.175 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:33:508.211 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:33:508.229 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:33:508.243 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:33:508.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:33:626.711 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:33:626.860 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:33:626.908 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:33:626.958 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:33:626.974 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:33:627.005 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:33:627.038 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:33:627.076 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:33:627.093 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:33:627.108 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:33:653.011 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:33:653.142 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:33:653.180 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:33:653.215 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:33:653.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:33:653.458 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:33:653.502 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:33:653.520 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:33:653.535 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:33:653.550 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:35:448.901 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-03-26 Thu 05:40:35:448.957 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:35:449.031 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503635448748 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 132 dequeues = 132 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 163 dequeues = 163 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774503612413052 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774503612413391 ] max_sm_queue_count = 5 ] 2026-03-26 Thu 05:40:35:449.046 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:35:449.056 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:35:449.231 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503635448884 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 62, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:35:449.262 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:38:508.272 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:38:508.476 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:38:508.543 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:38:508.634 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:38:508.690 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:38:508.770 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:38:508.807 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:38:508.830 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:38:508.851 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:38:508.872 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:38:627.011 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:38:627.273 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:38:627.311 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:38:627.355 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:38:627.410 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:38:627.455 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:38:627.498 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:38:627.518 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:38:627.533 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:38:627.547 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:38:653.568 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:38:653.763 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:38:653.825 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:38:653.861 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:38:653.898 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:38:654.181 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:38:654.219 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:38:654.245 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:38:654.268 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:38:654.284 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:42:415.908 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:40:42:417.958 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-26 Thu 05:40:42:418.106 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:40:42:418.145 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503642416025 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:40:42:418.163 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:40:42:418.174 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:40:42:418.345 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503642418077 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 64, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:42:418.363 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:42:418.619 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-26 Thu 05:40:42:418.652 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1):EvTcpConnectFail 2026-03-26 Thu 05:40:42:418.915 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-26 Thu 05:40:42:418.931 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Connect : EvTcpConnectFail 2026-03-26 Thu 05:40:42:419.056 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:40:42:419.091 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Idle 2026-03-26 Thu 05:40:42:419.162 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:42:419.337 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774503642419005 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:42:419.353 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:42:419.365 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:42:419.398 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774503642419034 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:40:42:419.411 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:40:42:419.442 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:40:42:419.479 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:40:42:419.651 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774503642419150 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:40:42:419.670 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:40:43:508.855 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:43:509.028 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:43:509.112 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:43:509.160 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:43:509.206 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:43:509.356 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:43:509.441 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:43:509.482 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:43:509.504 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:43:509.528 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:43:627.363 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:43:627.695 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:43:627.759 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:43:627.817 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:43:627.867 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:43:627.913 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:43:627.972 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:43:628.001 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:43:628.024 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:43:628.046 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:43:654.257 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:43:654.424 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:43:654.581 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:43:654.615 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:43:654.648 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:43:654.841 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:43:654.878 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:43:654.967 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:43:654.995 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:43:655.019 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:48:509.477 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:48:509.670 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:48:509.731 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:48:509.823 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:48:509.883 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:48:509.960 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:48:510.007 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:48:510.031 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:48:510.055 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:48:510.077 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:48:627.772 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:48:627.922 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:48:628.024 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:48:628.072 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:48:628.106 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:48:628.134 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:48:630.304 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:48:630.353 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:48:630.372 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:48:630.389 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:48:654.947 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:48:655.082 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:48:655.129 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:48:655.165 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:48:655.198 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:48:655.359 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:48:655.409 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:48:655.451 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:48:655.482 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:48:655.498 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:53:510.078 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:53:510.254 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:53:510.292 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:53:510.333 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:53:510.365 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:53:510.449 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:53:510.488 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:53:510.517 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:53:510.531 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:53:510.547 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:53:628.390 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:53:628.621 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:53:630.387 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:53:630.525 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:53:630.599 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:53:630.632 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:53:630.650 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:53:630.691 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:53:630.828 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:53:630.886 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:53:655.636 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:53:655.817 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:53:655.877 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:53:655.930 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:53:655.995 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:53:656.116 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:53:656.169 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:53:656.207 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:53:656.232 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:53:656.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:40:58:511.568 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:40:58:511.699 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:40:58:511.738 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:40:58:511.793 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:40:58:511.826 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:40:58:511.890 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:40:58:511.934 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:40:58:511.952 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:40:58:511.968 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:40:58:511.983 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:40:58:628.691 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:40:58:628.847 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:40:58:630.657 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:40:58:630.768 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:40:58:630.791 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:40:58:630.832 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:40:58:630.860 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:40:58:630.904 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:40:58:630.922 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:40:58:630.959 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:40:58:656.190 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:40:58:656.320 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:40:58:656.358 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:40:58:656.393 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:40:58:656.429 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:40:58:656.651 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:40:58:656.687 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:40:58:656.704 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:40:58:656.720 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:40:58:656.734 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:41:03:511.993 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:41:03:512.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:41:03:512.261 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:41:03:512.344 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:41:03:512.381 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:41:03:512.442 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:41:03:512.506 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:41:03:512.524 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:41:03:512.539 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:41:03:512.555 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:41:03:628.914 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:41:03:629.066 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:41:03:630.815 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:41:03:630.925 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:41:03:630.946 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:41:03:631.005 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:41:03:631.019 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:41:03:631.051 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:41:03:631.088 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:41:03:631.104 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:41:03:656.728 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:41:03:656.865 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:41:03:656.905 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:41:03:656.938 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:41:03:656.972 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:41:03:657.096 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:41:03:657.144 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:41:03:657.167 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:41:03:657.182 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:41:03:657.198 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:41:05:449.326 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:05:449.437 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: SANDESH: Send FAILED: 1774503665449191 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 139 dequeues = 139 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 173 dequeues = 173 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774503642419089 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774503642419477 ] max_sm_queue_count = 5 ] 2026-03-26 Thu 05:41:05:449.464 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:05:449.475 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:05:449.703 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: SANDESH: Send FAILED: 1774503665449302 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 69, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:05:449.724 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453091940096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:08:512.542 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:41:08:512.686 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:41:08:512.727 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:41:08:512.760 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:41:08:512.815 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:41:08:512.873 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1408fe0 2026-03-26 Thu 05:41:08:512.908 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1409920 2026-03-26 Thu 05:41:08:512.933 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140a260 2026-03-26 Thu 05:41:08:512.948 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140aba0 2026-03-26 Thu 05:41:08:512.963 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x140b4e0 2026-03-26 Thu 05:41:08:629.141 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:41:08:629.299 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405870 2026-03-26 Thu 05:41:08:631.154 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:41:08:631.250 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:41:08:631.287 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:41:08:631.321 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:41:08:631.367 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1405340 2026-03-26 Thu 05:41:08:631.390 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x13fe1d0 2026-03-26 Thu 05:41:08:631.405 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1404ca0 2026-03-26 Thu 05:41:08:631.420 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14045f0 2026-03-26 Thu 05:41:08:657.179 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:41:08:657.344 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:41:08:657.387 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:41:08:657.423 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:41:08:657.455 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:41:08:657.653 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14061a0 2026-03-26 Thu 05:41:08:657.726 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407420 2026-03-26 Thu 05:41:08:657.745 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1406ae0 2026-03-26 Thu 05:41:08:657.759 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x14086a0 2026-03-26 Thu 05:41:08:657.774 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: -1 0x1407d60 2026-03-26 Thu 05:41:10:445.930 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:10:446.020 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503670445602 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-03-26 Thu 05:41:10:446.043 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-03-26 Thu 05:41:12:419.957 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:41:12:420.178 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Connect : Start Connect timer 10.0.0.242:8086 2026-03-26 Thu 05:41:12:420.298 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:41:12:420.332 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503672420034 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.242:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:41:12:420.343 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:41:12:420.353 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:41:12:420.539 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503672420285 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.242:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 72, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:12:420.565 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:12:423.035 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-26 Thu 05:41:12:423.062 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: -(-1):EvTcpConnectFail 2026-03-26 Thu 05:41:12:425.806 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-26 Thu 05:41:12:425.829 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Connect : EvTcpConnectFail 2026-03-26 Thu 05:41:12:426.033 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-26 Thu 05:41:12:426.075 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Idle 2026-03-26 Thu 05:41:12:426.221 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:12:426.441 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503672425960 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 74, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:12:426.467 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:12:426.480 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:12:426.528 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503672425991 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:41:12:426.539 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:41:12:426.550 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:41:12:426.561 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:12:426.736 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503672426206 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 74, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:12:426.771 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:13:513.298 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1408fe0 2026-03-26 Thu 05:41:13:513.419 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1409920 2026-03-26 Thu 05:41:13:513.457 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140a260 2026-03-26 Thu 05:41:13:513.489 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140aba0 2026-03-26 Thu 05:41:13:513.570 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x140b4e0 2026-03-26 Thu 05:41:13:513.625 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1408fe0 2026-03-26 Thu 05:41:13:513.646 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:513.674 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:513.687 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1409920 2026-03-26 Thu 05:41:13:513.694 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:513.703 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:513.712 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140a260 2026-03-26 Thu 05:41:13:513.718 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:513.727 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:513.759 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140aba0 2026-03-26 Thu 05:41:13:513.765 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:513.774 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:513.794 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x140b4e0 2026-03-26 Thu 05:41:13:513.800 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:513.810 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:514.161 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:514.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:514.199 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:514.210 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:514.224 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:514.234 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:514.248 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:514.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:514.272 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:514.282 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:514.310 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:514.868 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:13:514.906 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503673514344 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.242:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:41:13:514.924 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:41:13:522.156 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:522.220 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:522.240 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:522.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:629.368 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405870 2026-03-26 Thu 05:41:13:629.552 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1405870 2026-03-26 Thu 05:41:13:629.565 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:629.591 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:629.715 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:629.731 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:629.759 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:631.411 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1405340 2026-03-26 Thu 05:41:13:631.648 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1405340 2026-03-26 Thu 05:41:13:631.664 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:631.684 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:631.695 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x13fe1d0 2026-03-26 Thu 05:41:13:631.757 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1404ca0 2026-03-26 Thu 05:41:13:631.800 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14045f0 2026-03-26 Thu 05:41:13:631.870 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x13fe1d0 2026-03-26 Thu 05:41:13:631.879 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:631.901 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:631.911 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1404ca0 2026-03-26 Thu 05:41:13:631.917 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:631.927 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:631.936 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14045f0 2026-03-26 Thu 05:41:13:631.942 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:631.951 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:631.982 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:631.994 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:632.122 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:13:632.156 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: SANDESH: Send FAILED: 1774503673632065 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.242:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:41:13:632.165 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453109397248, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:41:13:632.839 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:632.876 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:632.888 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:632.903 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:632.922 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:632.936 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:632.946 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:632.967 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:632.984 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:633.000 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:657.716 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14061a0 2026-03-26 Thu 05:41:13:657.852 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407420 2026-03-26 Thu 05:41:13:657.892 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1406ae0 2026-03-26 Thu 05:41:13:657.926 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x14086a0 2026-03-26 Thu 05:41:13:657.957 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1407d60 2026-03-26 Thu 05:41:13:658.080 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14061a0 2026-03-26 Thu 05:41:13:658.107 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:658.138 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:658.175 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1407420 2026-03-26 Thu 05:41:13:658.182 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:658.193 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:658.203 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1406ae0 2026-03-26 Thu 05:41:13:658.209 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:658.218 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:658.235 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x14086a0 2026-03-26 Thu 05:41:13:658.241 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:658.261 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:658.290 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: RAC_Connect status: 0 0x1407d60 2026-03-26 Thu 05:41:13:658.297 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:41:13:658.306 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-26 Thu 05:41:13:658.429 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:658.443 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:658.468 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:658.511 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:658.523 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:658.596 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:658.615 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:658.625 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:658.639 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:658.649 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:658.666 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:658.680 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:658.695 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-26 Thu 05:41:13:658.705 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-26 Thu 05:41:13:658.725 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-26 Thu 05:41:13:660.417 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453105198848, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:13:660.475 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453105198848, Pid 1]: SANDESH: Send FAILED: 1774503673658564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:41:13:660.487 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453105198848, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:41:35:449.566 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-03-26 Thu 05:41:35:449.621 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:35:449.699 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503695449431 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 150 dequeues = 150 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 187 dequeues = 187 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774503672426074 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774503673660400 ] max_sm_queue_count = 5 ] 2026-03-26 Thu 05:41:35:449.714 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:35:449.726 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:41:35:449.882 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503695449541 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 233, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:35:449.906 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:42:429.445 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-26 Thu 05:41:42:429.737 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-26 Thu 05:41:42:429.868 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:41:42:429.903 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: SANDESH: Send FAILED: 1774503702429554 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:41:42:429.913 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:41:42:429.924 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:41:42:430.110 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: SANDESH: Send FAILED: 1774503702429849 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 233, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 82, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:42:430.129 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452980299520, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:41:42:435.281 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:56086::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-26 Thu 05:41:42:435.321 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:56086::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:41:42:435.335 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453261465728, Pid 1]: 10.0.0.38:56086::10.0.0.38:8086(-1):EvTcpConnected 2026-03-26 Thu 05:41:42:444.267 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:41:42:444.318 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:41:42:444.375 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: ClientInit 2026-03-26 Thu 05:41:42:444.392 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 6 2026-03-26 Thu 05:41:42:444.501 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:41:42:444.522 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:41:42:444.551 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503702444364 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:41:42:444.560 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:41:42:444.570 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:41:42:444.579 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:41:42:444.592 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:41:42:444.599 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:41:42:444.755 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: SANDESH: Send FAILED: 1774503702444490 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 84, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-26 Thu 05:41:42:444.784 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453001287424, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:41:42:458.611 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: OnMessage control in state: ClientInit 2026-03-26 Thu 05:41:42:458.677 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-26 Thu 05:41:42:458.688 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-26 Thu 05:41:42:458.719 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Received Ctrl Message with size 0 2026-03-26 Thu 05:41:42:458.974 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 569219 calls = 69 average_bytes = 8249 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 150 dequeues = 150 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 187 dequeues = 187 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1774503672426074 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1774503673660400 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 61 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 534615 calls = 60 average_bytes = 8910 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 44 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 84, _iter59->first = sent _iter59->second = 70, ] ] max_sm_queue_count = 5 proxy seq 92 2026-03-26 Thu 05:41:42:459.007 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:41:42:459.039 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 45 2026-03-26 Thu 05:41:42:459.053 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:41:42:459.090 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 13 2026-03-26 Thu 05:41:42:459.101 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:41:42:459.110 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:41:42:459.145 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Established 2026-03-26 Thu 05:41:45:238.388 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: OnMessage control in state: Established 2026-03-26 Thu 05:41:45:238.501 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-03-26 Thu 05:41:45:238.514 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Established : EvSandeshMessageRecv 2026-03-26 Thu 05:41:45:238.564 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: Received Ctrl Message with size 3 2026-03-26 Thu 05:41:45:238.579 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 93, total = 0 2026-03-26 Thu 05:41:45:238.591 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 47, total = 0 2026-03-26 Thu 05:41:45:238.603 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 13, total = 0 2026-03-26 Thu 05:41:45:238.611 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140451940136704, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:42:35:451.452 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140452975179520, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-03-26 Thu 05:43:35:452.590 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 15 without seqno , total = 0 2026-03-26 Thu 05:44:35:454.113 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 16 without seqno , total = 0 2026-03-26 Thu 05:44:47:090.422 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvStop in state Established 2026-03-26 Thu 05:44:47:090.563 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Idle 2026-03-26 Thu 05:44:47:090.673 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-26 Thu 05:44:47:090.686 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-26 Thu 05:44:47:090.852 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: SANDESH: Send FAILED: 1774503887090660 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1774502895066362 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.242:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 4895 calls = 8 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 768339 calls = 88 average_bytes = 8731 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 15 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 58 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 86, _iter59->first = sent _iter59->second = 92, ] ] ] 2026-03-26 Thu 05:44:47:090.878 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 140453113595648, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:45:15:021.622 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: http-server-port 8091 2026-03-26 Thu 05:45:15:021.709 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Max-tasks 16 2026-03-26 Thu 05:45:15:021.717 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Max-slice 100 2026-03-26 Thu 05:45:15:021.723 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Collectors 10.0.0.38:8086 2026-03-26 Thu 05:45:15:021.729 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Collectors 10.0.0.242:8086 2026-03-26 Thu 05:45:15:021.734 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Collectors 10.0.0.33:8086 2026-03-26 Thu 05:45:15:021.775 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: ROLE : Generator 2026-03-26 Thu 05:45:15:021.783 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-03-26 Thu 05:45:15:021.788 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5546-1. 2026-03-26 Thu 05:45:15:021.794 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: NODE TYPE : Database 2026-03-26 Thu 05:45:15:021.800 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: INSTANCE ID : 0 2026-03-26 Thu 05:45:15:021.806 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-03-26 Thu 05:45:15:021.812 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-03-26 Thu 05:45:15:030.432 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: HTTP Introspect Init 2026-03-26 Thu 05:45:15:030.584 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-03-26 Thu 05:45:15:030.684 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Sandesh Http Server Port: 8091 2026-03-26 Thu 05:45:15:030.757 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-03-26 Thu 05:45:15:030.767 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-03-26 Thu 05:45:15:031.192 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Idle 2026-03-26 Thu 05:45:15:031.366 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: No Client: 1774503915031301 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774503915031205 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:45:15:033.744 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-03-26 Thu 05:45:15:033.819 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-03-26 Thu 05:45:15:033.835 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.242:9042 10.0.0.33:9042 2026-03-26 Thu 05:45:15:033.799 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvStart in state Idle 2026-03-26 Thu 05:45:15:033.960 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Disconnect 2026-03-26 Thu 05:45:15:034.049 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-03-26 Thu 05:45:15:034.474 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-26 Thu 05:45:15:034.626 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:45:15:034.660 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: SANDESH: Send FAILED: 1774503915033953 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:45:15:034.684 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:45:15:034.700 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:45:15:034.828 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: SANDESH: Send FAILED: 1774503915034034 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774503915031205 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:45:15:034.848 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:45:15:034.864 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:45:15:034.959 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: SANDESH: Send FAILED: 1774503915034113 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774503915031205 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.242:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:45:15:034.981 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:45:15:034.997 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:45:15:035.020 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: SANDESH: Send FAILED: 1774503915034134 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:45:15:035.032 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:45:15:035.045 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:45:15:035.144 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: SANDESH: Send FAILED: 1774503915034586 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1774503915031205 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.242:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:45:15:035.166 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-26 Thu 05:45:15:116.045 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-03-26 Thu 05:45:15:119.849 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:39440::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-03-26 Thu 05:45:15:119.883 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: 10.0.0.38:39440::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-26 Thu 05:45:15:119.892 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: 10.0.0.38:39440::10.0.0.38:8086(-1):EvTcpConnected 2026-03-26 Thu 05:45:15:119.920 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1251540 2026-03-26 Thu 05:45:15:119.926 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:119.946 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:119.962 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1256540 2026-03-26 Thu 05:45:15:119.967 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:119.977 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:119.994 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1256df0 2026-03-26 Thu 05:45:15:120.001 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.013 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.029 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1257a90 2026-03-26 Thu 05:45:15:120.038 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.046 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.057 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1258750 2026-03-26 Thu 05:45:15:120.063 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.070 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.080 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1259410 2026-03-26 Thu 05:45:15:120.085 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.092 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.116 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x1259fd0 2026-03-26 Thu 05:45:15:120.121 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.129 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.139 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125ac10 2026-03-26 Thu 05:45:15:120.145 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.152 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.163 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125b850 2026-03-26 Thu 05:45:15:120.176 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.194 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125c490 2026-03-26 Thu 05:45:15:120.199 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.206 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.242 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.217 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125d0d0 2026-03-26 Thu 05:45:15:120.222 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.230 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.241 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125dd10 2026-03-26 Thu 05:45:15:120.246 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.253 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.263 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125e950 2026-03-26 Thu 05:45:15:120.269 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.276 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.286 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x125f590 2026-03-26 Thu 05:45:15:120.291 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.299 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.308 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: RAC_Connect status: 0 0x12601d0 2026-03-26 Thu 05:45:15:120.314 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: Connected to REDIS... 2026-03-26 Thu 05:45:15:120.321 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-26 Thu 05:45:15:120.541 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.553 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.572 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.586 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.629 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.658 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.671 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.680 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.692 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.700 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.712 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.721 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.739 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.752 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.767 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.785 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.794 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.806 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.814 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.834 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.860 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:120.873 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.882 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.894 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.902 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.930 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.940 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.951 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.959 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.971 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:120.980 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:120.991 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:121.003 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:121.021 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:121.033 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:121.057 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.080 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.096 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.115 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.126 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.141 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.151 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.164 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-26 Thu 05:45:15:121.173 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.242:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-26 Thu 05:45:15:121.185 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-26 Thu 05:45:15:121.310 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-26 Thu 05:45:15:132.841 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915118057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-26 Thu 05:45:15:132.867 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-26 Thu 05:45:15:132.881 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-26 Thu 05:45:15:132.889 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-26 Thu 05:45:15:132.928 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit 2026-03-26 Thu 05:45:15:132.952 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5546-1.:contrail-query-engine:0:Database count 1 2026-03-26 Thu 05:45:15:133.046 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.056 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.074 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915120621 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-26 Thu 05:45:15:133.088 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.107 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.114 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.128 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915120652 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.242:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:45:15:133.136 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.144 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.151 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.164 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915120855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.242:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:45:15:133.171 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.178 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.187 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.200 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915121075 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:45:15:133.206 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.214 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.220 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.232 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915132921 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-26 Thu 05:45:15:133.241 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-26 Thu 05:45:15:133.249 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.256 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-26 Thu 05:45:15:133.280 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-26 Thu 05:45:15:133.287 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-26 Thu 05:45:15:133.395 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SANDESH: Send FAILED: 1774503915133034 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1774503915031205 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.242:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 63, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-26 Thu 05:45:15:133.421 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-26 Thu 05:45:15:136.072 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: OnMessage control in state: ClientInit 2026-03-26 Thu 05:45:15:136.133 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-26 Thu 05:45:15:136.151 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-26 Thu 05:45:15:136.171 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: Received Ctrl Message with size 0 2026-03-26 Thu 05:45:15:136.272 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1774503915031205 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.242:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 63, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2026-03-26 Thu 05:45:15:136.300 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:45:15:136.325 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5546-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.242:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.242:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 8 2026-03-26 Thu 05:45:15:136.337 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-26 Thu 05:45:15:136.344 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:45:15:136.351 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-26 Thu 05:45:15:136.394 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: Established 2026-03-26 Thu 05:46:15:031.879 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832240092736, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-03-26 Thu 05:47:15:033.227 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832202319424, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-03-26 Thu 05:48:15:033.761 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-03-26 Thu 05:49:15:034.395 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832335496768, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-03-26 Thu 05:50:15:034.944 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-03-26 Thu 05:51:15:035.533 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831518688832, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-03-26 Thu 05:52:15:036.283 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832331298368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-03-26 Thu 05:53:15:036.843 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-03-26 Thu 05:54:15:037.409 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832206517824, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-03-26 Thu 05:55:15:037.961 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831510292032, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-03-26 Thu 05:56:15:038.612 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832331298368, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-03-26 Thu 05:57:15:039.427 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-03-26 Thu 05:58:15:040.002 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832244291136, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-03-26 Thu 05:59:15:042.632 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-03-26 Thu 06:00:15:043.270 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 15 without seqno , total = 0 2026-03-26 Thu 06:01:15:043.791 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 16 without seqno , total = 0 2026-03-26 Thu 06:02:15:044.441 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832202319424, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 17 without seqno , total = 0 2026-03-26 Thu 06:03:15:044.930 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832248489536, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 18 without seqno , total = 0 2026-03-26 Thu 06:04:15:045.430 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 19 without seqno , total = 0 2026-03-26 Thu 06:04:59:041.269 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:45104 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-26 Thu 06:05:01:927.930 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:45110 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-26 Thu 06:05:15:046.059 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832206517824, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 20 without seqno , total = 0 2026-03-26 Thu 06:06:15:046.699 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 21 without seqno , total = 0 2026-03-26 Thu 06:06:44:554.714 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:43034 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-26 Thu 06:07:07:724.223 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-03-26 Thu 06:07:07:724.489 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID 05023822-28da-11f1-ba34-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-03-26 Thu 06:07:07:724.546 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 :0:json query is: { end_time: 1774505227651908 , enqueue_time: 1774505227723795 , query_metadata: {"enqueue_time": 1774505227723610} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1774505209384420 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-26 Thu 06:07:07:724.734 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-03-26 Thu 06:07:07:724.751 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-03-26 Thu 06:07:07:724.803 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for 05023822-28da-11f1-ba34-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-03-26 Thu 06:07:07:724.888 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 :0:json query is: { end_time: 1774505227651908 , enqueue_time: 1774505227723795 , query_metadata: {"enqueue_time": 1774505227723610} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1774505209384420 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-26 Thu 06:07:07:818.498 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:818.550 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:818.559 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:818.567 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:818.575 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:818.582 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:818.590 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:818.597 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:818.604 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:818.611 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:819.847 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:819.874 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:819.883 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:819.900 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:819.909 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:819.916 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:819.923 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:819.978 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:819.989 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 11 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:820.005 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:820.276 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:820.409 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:820.524 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:820.635 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:820.740 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:820.852 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:820.954 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:821.082 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:821.197 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:821.302 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:821.408 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:822.201 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:822.242 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:822.251 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:822.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:822.266 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:822.273 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:822.288 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:822.296 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:822.303 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 UNION between tables of sizes 0 and 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:07:822.422 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QELog: 05023822-28da-11f1-ba34-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:07:822.517 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 :0:json query is: { end_time: 1774505227651908 , enqueue_time: 1774505227723795 , query_metadata: {"enqueue_time": 1774505227723610} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1774505209384420 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-26 Thu 06:07:07:822.724 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:822.871 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:822.990 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.119 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.248 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.363 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.474 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.585 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.694 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.805 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:823.914 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.023 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.139 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.249 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.356 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.462 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.568 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.677 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.781 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.893 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:824.998 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.115 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.222 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.326 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.428 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.545 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.673 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.789 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:825.895 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:826.014 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:827.173 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:827.312 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:827.419 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832198121024, Pid 1]: [SYS_INFO]: QEQueryLog: 05023822-28da-11f1-ba34-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:07:827.841 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 44 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-03-26 Thu 06:07:07:828.587 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 44 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-03-26 Thu 06:07:07:828.804 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":6, "count":44} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-03-26 Thu 06:07:07:829.114 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5546-1. table = ObjectVNTable query_stats= [ qid = 05023822-28da-11f1-ba34-00000a000026 where = [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-11157719:ctest-vn22-93015443", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 94,0, 92,0, 97,0, chunk_select_time = 0,1, 0,0, 0,4, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 104 rows = 44 enq_delay = 0 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-03-26 Thu 06:07:07:830.070 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: Finished: QID 05023822-28da-11f1-ba34-00000a000026 Table ObjectVNTable Time(ms) 104 RedisTime(ms) 0 MergeTime(ms) 0 Rows 44 EnQ-delay0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-03-26 Thu 06:07:07:830.234 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-03-26 Thu 06:07:10:874.084 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-03-26 Thu 06:07:10:874.473 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID 06e2d9ac-28da-11f1-955e-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-03-26 Thu 06:07:10:874.538 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 :0:json query is: { end_time: 1774505230857857 , enqueue_time: 1774505230873691 , query_metadata: {"enqueue_time": 1774505230873577} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1774505209384420 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "8377297f-b102-41a2-85b5-33f59253c982", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-26 Thu 06:07:10:874.788 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-03-26 Thu 06:07:10:874.817 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-03-26 Thu 06:07:10:874.884 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for 06e2d9ac-28da-11f1-955e-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-03-26 Thu 06:07:10:874.976 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832339695168, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 :0:json query is: { end_time: 1774505230857857 , enqueue_time: 1774505230873691 , query_metadata: {"enqueue_time": 1774505230873577} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1774505209384420 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "8377297f-b102-41a2-85b5-33f59253c982", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-26 Thu 06:07:10:994.985 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.051 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.074 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.083 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.103 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.111 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.118 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.125 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.133 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.140 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.167 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.175 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.183 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.190 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.197 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.205 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.212 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.219 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.226 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:995.233 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:995.462 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 :0:json query is: { end_time: 1774505230857857 , enqueue_time: 1774505230873691 , query_metadata: {"enqueue_time": 1774505230873577} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1774505209384420 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "8377297f-b102-41a2-85b5-33f59253c982", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-26 Thu 06:07:10:999.813 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:999.853 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:999.862 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:999.869 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:999.876 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:999.884 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:999.891 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:999.898 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:10:999.905 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 UNION between tables of sizes 0 and 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-26 Thu 06:07:10:999.925 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QELog: 06e2d9ac-28da-11f1-955e-00000a000026 Resulting size of set 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-26 Thu 06:07:11:000.156 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 8377297f-b102-41a2-85b5-33f59253c982 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:11:000.284 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 8377297f-b102-41a2-85b5-33f59253c982 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:11:000.385 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 8377297f-b102-41a2-85b5-33f59253c982 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:11:000.489 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 8377297f-b102-41a2-85b5-33f59253c982 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:11:000.592 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: [SYS_INFO]: QEQueryLog: 06e2d9ac-28da-11f1-955e-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 8377297f-b102-41a2-85b5-33f59253c982 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-26 Thu 06:07:11:000.807 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 5 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-03-26 Thu 06:07:11:001.021 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 5 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-03-26 Thu 06:07:11:001.281 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":1, "count":5} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-03-26 Thu 06:07:11:001.436 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832206517824, Pid 1]: [SYS_INFO]: QELog: Finished: QID 06e2d9ac-28da-11f1-955e-00000a000026 Table ObjectVMTable Time(ms) 126 RedisTime(ms) 0 MergeTime(ms) 0 Rows 5 EnQ-delay0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-03-26 Thu 06:07:11:001.469 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831514490432, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5546-1. table = ObjectVMTable query_stats= [ qid = 06e2d9ac-28da-11f1-955e-00000a000026 where = [[{"name": "ObjectId", "value": "8377297f-b102-41a2-85b5-33f59253c982", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 119,0, 120,0, 124,0, chunk_select_time = 0,0, 0,0, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 126 rows = 5 enq_delay = 0 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-03-26 Thu 06:07:11:001.500 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832206517824, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-03-26 Thu 06:07:15:047.369 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832206517824, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 22 without seqno , total = 0 2026-03-26 Thu 06:08:15:047.950 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 23 without seqno , total = 0 2026-03-26 Thu 06:09:15:048.497 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 24 without seqno , total = 0 2026-03-26 Thu 06:10:15:049.355 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 25 without seqno , total = 0 2026-03-26 Thu 06:11:15:050.788 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 26 without seqno , total = 0 2026-03-26 Thu 06:12:15:051.344 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831518688832, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 27 without seqno , total = 0 2026-03-26 Thu 06:13:15:051.950 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831518688832, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 28 without seqno , total = 0 2026-03-26 Thu 06:14:15:052.549 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 29 without seqno , total = 0 2026-03-26 Thu 06:15:15:053.085 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832252687936, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 30 without seqno , total = 0 2026-03-26 Thu 06:16:15:053.634 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 31 without seqno , total = 0 2026-03-26 Thu 06:17:15:054.742 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832339695168, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 32 without seqno , total = 0 2026-03-26 Thu 06:18:15:055.301 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832210716224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 33 without seqno , total = 0 2026-03-26 Thu 06:19:15:056.409 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139831506093632, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 34 without seqno , total = 0 2026-03-26 Thu 06:19:37:384.922 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:40210 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-26 Thu 06:20:15:059.400 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832339695168, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 35 without seqno , total = 0 2026-03-26 Thu 06:21:15:060.258 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832339695168, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 36 without seqno , total = 0 2026-03-26 Thu 06:22:15:060.888 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832256886336, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 37 without seqno , total = 0 2026-03-26 Thu 06:22:16:675.215 UTC cn-jenkins-deploy-platform-ansible-os-5546-1 [Thread 139832346835072, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:33602 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256