2026-06-04 Thu 05:58:36:523.933 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: http-server-port 8091 2026-06-04 Thu 05:58:36:524.030 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Max-tasks 16 2026-06-04 Thu 05:58:36:524.040 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Max-slice 100 2026-06-04 Thu 05:58:36:524.049 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Collectors 10.0.0.38:8086 2026-06-04 Thu 05:58:36:524.067 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Collectors 10.0.0.50:8086 2026-06-04 Thu 05:58:36:524.074 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Collectors 10.0.0.33:8086 2026-06-04 Thu 05:58:36:524.107 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: ROLE : Generator 2026-06-04 Thu 05:58:36:524.116 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-04 Thu 05:58:36:524.122 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5888-1. 2026-06-04 Thu 05:58:36:524.130 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-04 Thu 05:58:36:524.136 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-04 Thu 05:58:36:524.143 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-04 Thu 05:58:36:524.150 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-04 Thu 05:58:36:528.634 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: HTTP Introspect Init 2026-06-04 Thu 05:58:36:529.289 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-04 Thu 05:58:36:529.415 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Sandesh Http Server Port: 8091 2026-06-04 Thu 05:58:36:529.508 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-04 Thu 05:58:36:529.521 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-04 Thu 05:58:36:530.073 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Idle 2026-06-04 Thu 05:58:36:530.277 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: No Client: 1780552716530185 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:58:36:530.452 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-04 Thu 05:58:36:530.473 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-04 Thu 05:58:36:530.489 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-04 Thu 05:58:36:530.553 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvStart in state Idle 2026-06-04 Thu 05:58:36:530.645 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Disconnect 2026-06-04 Thu 05:58:36:530.718 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-04 Thu 05:58:36:531.000 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-04 Thu 05:58:36:531.092 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:58:36:531.116 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552716530636 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-04 Thu 05:58:36:531.126 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:58:36:531.136 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:58:36:531.218 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552716530704 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:58:36:531.237 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:58:36:531.247 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:58:36:531.470 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552716530758 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:58:36:531.491 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:58:36:531.502 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:58:36:531.521 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552716530793 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-04 Thu 05:58:36:531.528 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:58:36:531.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:58:36:531.598 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552716531076 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:58:36:531.613 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:58:36:536.106 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-04 Thu 05:58:36:536.146 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:58:36:536.221 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 0 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:58:36:538.837 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:58:36:538.882 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: SANDESH: Send FAILED: 1780552716536191 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-04 Thu 05:58:36:538.891 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:58:41:538.686 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-04 Thu 05:58:41:538.733 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:58:41:538.757 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 1 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:58:46:540.767 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-04 Thu 05:58:46:540.811 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:58:46:540.829 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 2 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:58:51:542.377 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-06-04 Thu 05:58:51:542.424 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:58:51:542.442 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 3 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:58:56:609.650 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-04 Thu 05:58:56:609.703 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:58:56:609.721 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 4 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:59:01:684.199 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-04 Thu 05:59:01:684.266 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:59:01:684.292 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 5 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:59:06:742.489 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-04 Thu 05:59:06:742.592 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:59:06:742.630 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 6 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:59:11:807.451 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1003 2026-06-04 Thu 05:59:11:807.544 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:59:11:807.563 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 7 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:59:16:849.558 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-06-04 Thu 05:59:16:849.607 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 8 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-06-04 Thu 05:59:21:883.098 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-04 Thu 05:59:31:892.095 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-04 Thu 05:59:31:892.156 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1):EvTcpConnectFail 2026-06-04 Thu 05:59:31:892.184 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2986180 2026-06-04 Thu 05:59:31:892.193 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:892.210 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:892.224 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298b180 2026-06-04 Thu 05:59:31:892.230 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:892.240 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:892.251 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298b830 2026-06-04 Thu 05:59:31:892.258 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:892.266 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:892.278 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298c130 2026-06-04 Thu 05:59:31:892.284 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:892.294 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:892.305 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298cc50 2026-06-04 Thu 05:59:31:892.311 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:892.320 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:892.335 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:59:31:892.405 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780552771891910 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-04 Thu 05:59:31:892.418 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:31:892.432 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-04 Thu 05:59:31:892.441 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Connect : EvTcpConnectFail 2026-06-04 Thu 05:59:31:892.611 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 05:59:31:892.668 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Idle 2026-06-04 Thu 05:59:31:892.743 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:892.857 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780552771892543 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 33, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:59:31:892.884 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:59:31:892.898 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:892.918 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780552771892580 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-04 Thu 05:59:31:892.927 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:31:892.938 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 05:59:31:892.994 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:893.097 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780552771892728 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 34, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:59:31:893.124 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:59:31:892.342 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298e290 2026-06-04 Thu 05:59:31:901.868 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:901.901 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:901.927 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298d770 2026-06-04 Thu 05:59:31:901.934 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:901.943 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:901.955 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298ee50 2026-06-04 Thu 05:59:31:901.961 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:901.970 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:901.982 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2990ef0 2026-06-04 Thu 05:59:31:901.988 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:901.998 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.008 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298f930 2026-06-04 Thu 05:59:31:902.015 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:902.024 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.035 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2990410 2026-06-04 Thu 05:59:31:902.041 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:902.051 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.074 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x29919d0 2026-06-04 Thu 05:59:31:902.081 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:902.091 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.114 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2993a70 2026-06-04 Thu 05:59:31:902.120 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:902.130 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.142 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2992f90 2026-06-04 Thu 05:59:31:902.149 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:902.158 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.167 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x29924b0 2026-06-04 Thu 05:59:31:902.173 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 05:59:31:902.182 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 05:59:31:902.400 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:902.455 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: SANDESH: Send FAILED: 1780552771902273 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780552771892666 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780552771892992 ] max_sm_queue_count = 5 ] 2026-06-04 Thu 05:59:31:902.464 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:59:31:902.471 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:902.553 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: SANDESH: Send FAILED: 1780552771902371 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 44, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:59:31:902.577 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:59:31:902.906 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:902.925 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:902.942 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:902.953 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:902.969 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:902.980 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.001 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.014 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.029 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.040 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.055 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.079 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.154 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:903.187 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:903.206 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:903.223 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:903.239 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:903.268 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:903.298 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.310 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.325 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.337 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.351 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.362 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.376 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.386 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.405 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.416 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.431 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.441 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.455 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.466 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.487 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:903.497 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:903.469 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:904.436 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: SANDESH: Send FAILED: 1780552771903130 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 05:59:31:904.452 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:31:904.468 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:904.494 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: SANDESH: Send FAILED: 1780552771903180 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 05:59:31:904.503 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:31:917.319 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.369 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.395 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.410 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.425 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.439 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.452 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.486 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.571 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 05:59:31:917.583 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 05:59:31:917.608 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 05:59:31:917.671 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:917.712 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552771917268 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 05:59:31:917.722 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:31:917.733 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 05:59:31:917.751 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780552771917388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 05:59:31:917.759 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:32:892.809 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 05:59:32:893.203 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-04 Thu 05:59:32:893.306 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:59:32:893.337 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: SANDESH: Send FAILED: 1780552772893011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 05:59:32:893.354 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 05:59:32:893.364 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 05:59:32:893.448 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: SANDESH: Send FAILED: 1780552772893291 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 89, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:59:32:893.465 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 05:59:32:897.607 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:37476::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 05:59:32:897.648 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:37476::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 05:59:32:897.656 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:37476::10.0.0.33:8086(-1):EvTcpConnected 2026-06-04 Thu 05:59:32:903.499 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 05:59:32:903.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 05:59:32:903.598 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: ClientInit 2026-06-04 Thu 05:59:32:903.623 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 1 2026-06-04 Thu 05:59:32:903.716 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 05:59:32:903.724 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 05:59:32:903.743 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: SANDESH: Send FAILED: 1780552772903588 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 05:59:32:903.761 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 05:59:32:903.769 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 05:59:32:903.774 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 05:59:32:903.785 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 05:59:32:903.790 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 05:59:32:903.872 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: SANDESH: Send FAILED: 1780552772903700 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 05:59:32:903.888 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 05:59:32:906.928 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: OnMessage control in state: ClientInit 2026-06-04 Thu 05:59:32:906.990 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-04 Thu 05:59:32:906.999 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-04 Thu 05:59:32:907.017 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Received Ctrl Message with size 0 2026-06-04 Thu 05:59:32:907.140 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780552771892666 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780552771892992 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 10 2026-06-04 Thu 05:59:32:907.175 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 05:59:32:907.195 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 11 2026-06-04 Thu 05:59:32:907.207 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 05:59:32:907.214 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-04 Thu 05:59:32:907.220 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-04 Thu 05:59:32:907.255 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Established 2026-06-04 Thu 05:59:37:381.360 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: OnMessage control in state: Established 2026-06-04 Thu 05:59:37:381.439 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-06-04 Thu 05:59:37:381.470 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Established : EvSandeshMessageRecv 2026-06-04 Thu 05:59:37:381.514 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Received Ctrl Message with size 3 2026-06-04 Thu 05:59:37:381.531 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 11, total = 0 2026-06-04 Thu 05:59:37:381.543 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 13, total = 0 2026-06-04 Thu 05:59:37:381.555 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 1, total = 0 2026-06-04 Thu 05:59:37:381.562 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-04 Thu 06:00:01:902.757 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-04 Thu 06:01:01:903.544 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-04 Thu 06:02:01:904.375 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-04 Thu 06:03:01:905.054 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-04 Thu 06:03:03:925.504 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:52718 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-04 Thu 06:03:17:674.798 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:36080 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-04 Thu 06:04:01:905.815 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638540105280, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-04 Thu 06:05:01:906.711 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-06-04 Thu 06:05:21:070.757 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2990410 2026-06-04 Thu 06:05:21:070.834 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:070.937 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298f930 2026-06-04 Thu 06:05:21:070.950 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:070.975 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-04 Thu 06:05:21:070.983 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298d770 2026-06-04 Thu 06:05:21:070.990 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:071.012 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298ee50 2026-06-04 Thu 06:05:21:071.019 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:071.038 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298e290 2026-06-04 Thu 06:05:21:071.045 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:071.328 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:05:21:071.754 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:05:21:071.779 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:05:21:071.793 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:05:21:071.805 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:05:21:086.359 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2993a70 2026-06-04 Thu 06:05:21:086.415 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:086.534 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x29924b0 2026-06-04 Thu 06:05:21:086.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:086.567 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x29919d0 2026-06-04 Thu 06:05:21:086.575 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:086.597 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2992f90 2026-06-04 Thu 06:05:21:086.604 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:086.647 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-04 Thu 06:05:21:086.654 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2990ef0 2026-06-04 Thu 06:05:21:086.661 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:05:21:088.094 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:05:21:088.207 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:05:21:088.229 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:05:21:088.245 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:05:21:088.261 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:05:26:071.402 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:05:26:081.816 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:05:26:081.889 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:05:26:081.930 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:05:26:081.971 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:05:26:082.243 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:05:26:082.298 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:05:26:082.327 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:05:26:082.341 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:05:26:082.356 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:05:26:088.246 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:05:26:088.410 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:05:26:088.447 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:05:26:088.483 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:05:26:088.525 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:05:26:088.758 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:05:26:088.820 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:05:26:088.833 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:05:26:088.845 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:05:26:088.859 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:05:31:082.326 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:05:31:086.275 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:05:31:086.325 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:05:31:086.354 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:05:31:086.382 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:05:31:086.582 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:05:31:086.628 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:05:31:086.641 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:05:31:086.654 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:05:31:086.700 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:05:31:088.837 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:05:31:091.056 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:05:31:091.818 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:05:31:091.867 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:05:31:091.895 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:05:31:091.934 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:05:31:092.010 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:05:31:092.032 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:05:31:092.053 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:05:31:092.079 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:05:31:208.337 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:37476::10.0.0.33:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-04 Thu 06:05:31:226.511 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:37476::10.0.0.33:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:05:31:226.544 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:37476::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-04 Thu 06:05:31:226.553 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:37476::10.0.0.33:8086(-1):EvTcpClose 2026-06-04 Thu 06:05:31:233.279 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvTcpClose in state Established 2026-06-04 Thu 06:05:31:233.323 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Established : EvTcpClose 2026-06-04 Thu 06:05:31:233.667 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-04 Thu 06:05:31:233.769 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:05:31:233.948 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780553131233420 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = cn-jenkins-deploy-platform-ansible-os-5888-3. collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:31:233.980 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:31:233.994 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:05:31:234.013 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780553131233462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:05:31:234.020 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:05:31:234.028 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-04 Thu 06:05:31:234.037 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:05:31:234.171 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: SANDESH: Send FAILED: 1780553131233749 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:31:234.199 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638172313152, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:31:234.332 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-04 Thu 06:05:31:234.346 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1):EvTcpConnectFail 2026-06-04 Thu 06:05:31:238.466 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-04 Thu 06:05:31:238.515 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Connect : EvTcpConnectFail 2026-06-04 Thu 06:05:31:238.688 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:05:31:239.999 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Idle 2026-06-04 Thu 06:05:31:240.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:05:31:240.458 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553131238608 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:31:240.521 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:31:240.538 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:05:31:240.592 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553131238657 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:05:31:240.603 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:05:31:240.615 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:05:31:240.692 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:05:31:240.921 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553131240155 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:31:241.144 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:31:907.182 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:05:31:907.345 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: SANDESH: Send FAILED: 1780553131906893 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 62 dequeues = 62 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 74 dequeues = 74 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553131239996 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553131240689 ] max_sm_queue_count = 5 ] 2026-06-04 Thu 06:05:31:907.366 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:31:907.379 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:05:31:907.533 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: SANDESH: Send FAILED: 1780553131907130 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:31:907.570 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:32:240.150 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:05:32:240.465 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-04 Thu 06:05:32:240.612 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:05:32:240.649 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SANDESH: Send FAILED: 1780553132240231 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:05:32:240.661 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:05:32:240.672 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:05:32:240.858 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SANDESH: Send FAILED: 1780553132240591 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:32:240.882 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:05:32:243.436 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:43868::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 06:05:32:243.471 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:43868::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:05:32:243.481 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:43868::10.0.0.38:8086(-1):EvTcpConnected 2026-06-04 Thu 06:05:32:243.719 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:05:32:243.753 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:05:32:243.810 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: ClientInit 2026-06-04 Thu 06:05:32:243.844 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 2 2026-06-04 Thu 06:05:32:243.959 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:05:32:243.986 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:05:32:244.015 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: SANDESH: Send FAILED: 1780553132243800 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:05:32:244.025 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:05:32:244.036 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:05:32:244.046 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:05:32:244.081 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:05:32:244.102 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:05:32:244.265 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: SANDESH: Send FAILED: 1780553132243933 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 108, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 36, ] ] ] 2026-06-04 Thu 06:05:32:244.296 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:05:32:251.274 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: OnMessage control in state: ClientInit 2026-06-04 Thu 06:05:32:251.372 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-04 Thu 06:05:32:251.396 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-04 Thu 06:05:32:251.427 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Received Ctrl Message with size 0 2026-06-04 Thu 06:05:32:251.660 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 311732 calls = 35 average_bytes = 8906 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 62 dequeues = 62 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 74 dequeues = 74 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553131239996 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553131240689 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 32 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 283428 calls = 31 average_bytes = 9142 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 24 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 108, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 36, ] ] max_sm_queue_count = 5 proxy seq 41 2026-06-04 Thu 06:05:32:251.708 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:05:32:251.736 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 19 2026-06-04 Thu 06:05:32:251.751 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:05:32:251.788 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.384009 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 6 2026-06-04 Thu 06:05:32:251.800 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:05:32:251.809 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-04 Thu 06:05:32:251.854 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Established 2026-06-04 Thu 06:05:36:086.665 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:05:36:086.851 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:05:36:086.897 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:05:36:086.931 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:05:36:086.967 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:05:36:087.123 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:05:36:087.154 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:05:36:087.173 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:05:36:087.217 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:05:36:087.235 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:05:36:091.994 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:05:36:092.138 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:05:36:092.184 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:05:36:092.222 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:05:36:092.261 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:05:36:092.413 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:05:36:092.443 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:05:36:092.529 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:05:36:092.570 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:05:36:092.586 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:05:41:087.373 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:05:41:087.499 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:05:41:087.534 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:05:41:087.561 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:05:41:087.591 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:05:41:089.605 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2990410 2026-06-04 Thu 06:05:41:089.634 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:089.682 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:089.696 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298f930 2026-06-04 Thu 06:05:41:089.701 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:089.708 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:089.716 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298d770 2026-06-04 Thu 06:05:41:089.721 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:089.728 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:089.736 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298e290 2026-06-04 Thu 06:05:41:089.740 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:089.747 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:089.754 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298ee50 2026-06-04 Thu 06:05:41:089.759 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:089.765 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:090.379 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:090.402 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:090.444 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:090.459 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:090.467 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:090.477 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:090.485 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:090.510 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:090.518 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:090.529 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:090.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:094.439 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:094.476 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:094.490 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:094.503 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:096.118 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:05:41:096.215 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:05:41:096.255 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:05:41:096.292 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:05:41:096.330 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:05:41:096.487 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2993a70 2026-06-04 Thu 06:05:41:096.497 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:096.511 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:096.540 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x29924b0 2026-06-04 Thu 06:05:41:096.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:096.553 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:096.609 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x29919d0 2026-06-04 Thu 06:05:41:096.616 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:096.624 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:096.659 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2992f90 2026-06-04 Thu 06:05:41:096.664 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:096.672 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:096.710 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2990ef0 2026-06-04 Thu 06:05:41:096.715 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:05:41:096.722 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:05:41:096.907 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:096.920 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:096.941 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:096.960 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:096.971 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:096.986 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:096.998 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:097.014 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:097.025 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:113.179 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:113.238 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:113.251 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:05:41:113.280 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:05:41:113.303 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:05:41:113.800 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:06:01:909.840 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-06-04 Thu 06:07:01:912.865 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638121940544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-06-04 Thu 06:07:43:977.569 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298ee50 2026-06-04 Thu 06:07:43:977.677 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:43:977.860 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298e290 2026-06-04 Thu 06:07:43:977.893 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:43:977.927 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298f930 2026-06-04 Thu 06:07:43:977.939 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:43:977.972 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-04 Thu 06:07:43:977.981 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298d770 2026-06-04 Thu 06:07:43:977.993 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:43:978.022 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2990410 2026-06-04 Thu 06:07:43:978.034 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:000.013 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:07:44:000.384 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:07:44:000.430 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:07:44:000.452 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:07:44:000.472 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:07:44:036.051 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-04 Thu 06:07:44:036.118 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2990ef0 2026-06-04 Thu 06:07:44:036.134 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:036.211 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2992f90 2026-06-04 Thu 06:07:44:036.227 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:036.255 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x29919d0 2026-06-04 Thu 06:07:44:036.267 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:036.295 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x29924b0 2026-06-04 Thu 06:07:44:036.308 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:036.606 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2993a70 2026-06-04 Thu 06:07:44:036.649 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:043.677 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:07:44:043.766 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:07:44:043.875 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:07:44:043.921 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:07:44:043.954 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:07:44:175.171 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298cc50 2026-06-04 Thu 06:07:44:175.238 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:175.363 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298c130 2026-06-04 Thu 06:07:44:175.384 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:175.422 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298b830 2026-06-04 Thu 06:07:44:175.431 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:175.746 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x298b180 2026-06-04 Thu 06:07:44:175.772 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:175.827 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-06-04 Thu 06:07:44:175.837 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Disconnect status: -1 0x2986180 2026-06-04 Thu 06:07:44:175.848 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-06-04 Thu 06:07:44:192.497 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:07:44:192.560 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:07:44:192.584 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:07:44:192.602 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:07:44:192.617 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:07:44:215.594 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:43868::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-04 Thu 06:07:44:215.634 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:43868::10.0.0.38:8086(-1):EvTcpClose 2026-06-04 Thu 06:07:44:224.083 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpClose in state Established 2026-06-04 Thu 06:07:44:224.137 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Established : EvTcpClose 2026-06-04 Thu 06:07:44:226.908 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-04 Thu 06:07:44:227.063 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:44:227.271 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553264224245 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = cn-jenkins-deploy-platform-ansible-os-5888-1. collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476795 calls = 55 average_bytes = 8669 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-04 Thu 06:07:44:230.375 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:44:230.412 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:44:230.452 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553264224295 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:44:230.462 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:44:230.475 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-04 Thu 06:07:44:230.487 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:44:230.596 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553264227037 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476795 calls = 55 average_bytes = 8669 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-04 Thu 06:07:44:230.618 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:44:234.022 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:51956::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 06:07:44:234.053 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:51956::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:07:44:234.065 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:51956::10.0.0.33:8086(-1):EvTcpConnected 2026-06-04 Thu 06:07:44:254.928 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:07:44:254.976 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:07:44:255.038 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit 2026-06-04 Thu 06:07:44:255.063 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 3 2026-06-04 Thu 06:07:44:255.203 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:44:255.219 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:07:44:255.252 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553264255026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:44:255.275 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:07:44:255.420 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:44:255.440 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:07:44:255.476 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:44:255.485 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:07:44:255.660 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553264255184 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 476795 calls = 55 average_bytes = 8669 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 59, ] ] ] 2026-06-04 Thu 06:07:44:255.686 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:07:44:289.310 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:51956::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-04 Thu 06:07:44:289.355 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:51956::10.0.0.33:8086(-1):EvTcpClose 2026-06-04 Thu 06:07:44:294.247 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-04 Thu 06:07:44:294.293 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: ClientInit : EvTcpClose 2026-06-04 Thu 06:07:44:294.428 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Return to idle with 1000 2026-06-04 Thu 06:07:44:294.438 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Idle 2026-06-04 Thu 06:07:44:294.493 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:44:294.654 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553264294371 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478368 calls = 56 average_bytes = 8542 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-04 Thu 06:07:44:294.683 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:44:294.695 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:44:294.712 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553264294405 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:44:294.722 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:44:294.730 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:07:44:294.738 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:44:294.827 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553264294476 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478368 calls = 56 average_bytes = 8542 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-04 Thu 06:07:44:294.844 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:45:297.827 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:07:45:298.166 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-04 Thu 06:07:45:298.319 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:45:298.356 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553265297930 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:45:298.367 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:45:298.380 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:45:298.548 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553265298298 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 3 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478368 calls = 56 average_bytes = 8542 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-04 Thu 06:07:45:298.575 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:45:305.626 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:49976::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 06:07:45:305.674 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:49976::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:07:45:305.686 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:49976::10.0.0.50:8086(-1):EvTcpConnected 2026-06-04 Thu 06:07:45:312.214 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:07:45:312.268 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:07:45:312.326 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: ClientInit 2026-06-04 Thu 06:07:45:312.349 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 4 2026-06-04 Thu 06:07:45:312.461 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:45:312.474 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:07:45:312.505 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553265312315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:45:312.516 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:07:45:312.527 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:45:312.536 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:07:45:312.553 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:45:312.561 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:07:45:312.735 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553265312444 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 478368 calls = 56 average_bytes = 8542 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 60, ] ] ] 2026-06-04 Thu 06:07:45:312.767 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:07:45:315.698 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:49976::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-04 Thu 06:07:45:315.737 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:49976::10.0.0.50:8086(-1):EvTcpClose 2026-06-04 Thu 06:07:45:319.213 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-04 Thu 06:07:45:319.245 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: ClientInit : EvTcpClose 2026-06-04 Thu 06:07:45:319.527 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Return to idle with 1000 2026-06-04 Thu 06:07:45:319.542 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Idle 2026-06-04 Thu 06:07:45:319.615 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:45:319.787 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: SANDESH: Send FAILED: 1780553265319442 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479941 calls = 57 average_bytes = 8420 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-04 Thu 06:07:45:319.824 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:45:319.838 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:45:319.865 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: SANDESH: Send FAILED: 1780553265319492 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:45:319.876 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:45:319.887 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:07:45:319.898 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:45:320.048 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: SANDESH: Send FAILED: 1780553265319599 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479941 calls = 57 average_bytes = 8420 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-04 Thu 06:07:45:320.069 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638544303680, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:46:322.136 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:07:46:322.447 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-04 Thu 06:07:46:322.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:46:322.562 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553266322220 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:46:322.569 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:46:322.577 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:46:322.685 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553266322524 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 4 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479941 calls = 57 average_bytes = 8420 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-04 Thu 06:07:46:322.707 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:46:326.233 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:34302::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 06:07:46:326.405 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:34302::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:07:46:326.416 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:34302::10.0.0.38:8086(-1):EvTcpConnected 2026-06-04 Thu 06:07:46:326.472 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:07:46:326.499 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:07:46:326.542 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit 2026-06-04 Thu 06:07:46:326.565 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 5 2026-06-04 Thu 06:07:46:326.650 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:46:326.659 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:07:46:326.675 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553266326535 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:46:326.693 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:07:46:326.700 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:46:326.706 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:07:46:326.716 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:46:326.722 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:07:46:326.812 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553266326638 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 479941 calls = 57 average_bytes = 8420 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 61, ] ] ] 2026-06-04 Thu 06:07:46:326.829 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:07:49:000.133 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:07:49:000.449 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:07:49:000.512 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:07:49:000.550 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:07:49:000.588 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:07:49:000.922 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:07:49:000.955 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:07:49:000.969 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:07:49:000.982 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:07:49:003.216 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:07:49:043.778 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:07:49:043.946 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:07:49:043.995 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:07:49:044.028 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:07:49:044.063 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:07:49:044.241 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:07:49:044.273 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:07:49:044.286 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:07:49:044.298 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:07:49:044.310 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:07:49:192.616 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:07:49:192.800 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:07:49:192.859 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:07:49:192.890 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:07:49:192.914 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:07:49:192.945 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:07:49:192.972 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:07:49:193.003 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:07:49:193.017 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:07:49:193.028 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:07:50:326.706 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: 10.0.0.38:8086 EvConnectTimerExpired in state ClientInit 2026-06-04 Thu 06:07:50:326.818 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-06-04 Thu 06:07:50:326.856 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: ClientInit : EvConnectTimerExpired 2026-06-04 Thu 06:07:50:327.158 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Return to idle with 1000 2026-06-04 Thu 06:07:50:327.174 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Idle 2026-06-04 Thu 06:07:50:327.260 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:50:327.455 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: SANDESH: Send FAILED: 1780553270326959 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 481514 calls = 58 average_bytes = 8301 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-04 Thu 06:07:50:327.492 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:50:327.504 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:50:327.523 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: SANDESH: Send FAILED: 1780553270327021 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:50:327.530 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:50:327.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:07:50:327.578 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:50:327.665 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: SANDESH: Send FAILED: 1780553270327243 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 481514 calls = 58 average_bytes = 8301 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-04 Thu 06:07:50:327.684 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638535906880, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:51:327.326 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:07:51:327.627 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-04 Thu 06:07:51:327.739 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:51:327.765 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553271327441 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:51:327.772 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:51:327.781 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:51:327.895 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553271327725 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 481514 calls = 58 average_bytes = 8301 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-04 Thu 06:07:51:327.923 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:51:329.494 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:52026::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 06:07:51:329.524 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:52026::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:07:51:329.532 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:52026::10.0.0.33:8086(-1):EvTcpConnected 2026-06-04 Thu 06:07:51:329.597 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:07:51:329.620 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:07:51:329.670 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: ClientInit 2026-06-04 Thu 06:07:51:329.695 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 6 2026-06-04 Thu 06:07:51:329.780 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:51:329.788 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:07:51:329.807 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553271329662 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:51:329.814 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:07:51:329.822 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:51:329.839 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:07:51:329.854 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:07:51:329.862 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:07:51:329.993 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553271329768 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 481514 calls = 58 average_bytes = 8301 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 56, _iter59->first = sent _iter59->second = 62, ] ] ] 2026-06-04 Thu 06:07:51:330.011 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:07:54:001.020 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:07:54:001.207 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:07:54:001.250 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:07:54:001.281 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:07:54:001.663 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:07:54:001.715 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:07:54:001.751 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:07:54:001.765 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:07:54:003.321 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:07:54:003.714 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:07:54:044.330 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:07:54:044.494 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:07:54:044.529 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:07:54:044.582 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:07:54:044.624 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:07:54:044.869 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:07:54:044.912 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:07:54:044.929 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:07:54:044.943 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:07:54:044.955 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:07:54:192.950 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:07:54:193.264 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:07:54:193.320 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:07:54:193.402 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:07:54:193.456 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:07:54:193.500 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:07:54:193.551 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:07:54:193.575 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:07:54:193.593 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:07:54:193.609 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:07:56:080.982 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:52026::10.0.0.33:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-04 Thu 06:07:56:081.052 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:52026::10.0.0.33:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:07:56:081.075 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:52026::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-04 Thu 06:07:56:081.083 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:52026::10.0.0.33:8086(-1):EvTcpClose 2026-06-04 Thu 06:07:56:081.847 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-06-04 Thu 06:07:56:081.870 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: ClientInit : EvTcpClose 2026-06-04 Thu 06:07:56:082.049 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Return to idle with 1000 2026-06-04 Thu 06:07:56:082.064 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Idle 2026-06-04 Thu 06:07:56:082.133 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:56:082.273 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553276081959 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:07:56:082.294 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:56:082.308 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:56:082.327 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553276081996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:56:082.334 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:56:082.342 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:07:56:082.350 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:56:082.425 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553276082119 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:07:56:082.449 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:57:082.968 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:07:57:083.269 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-04 Thu 06:07:57:083.401 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:57:083.431 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: SANDESH: Send FAILED: 1780553277083072 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:57:083.440 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:57:083.449 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:07:57:083.574 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: SANDESH: Send FAILED: 1780553277083381 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:07:57:083.599 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:57:085.157 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-04 Thu 06:07:57:085.180 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1):EvTcpConnectFail 2026-06-04 Thu 06:07:57:086.410 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-04 Thu 06:07:57:086.437 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Connect : EvTcpConnectFail 2026-06-04 Thu 06:07:57:086.618 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:07:57:086.664 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Idle 2026-06-04 Thu 06:07:57:086.717 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:57:086.828 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553277086540 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:07:57:086.856 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:57:086.869 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:57:086.893 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553277086583 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:07:57:086.902 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:07:57:086.916 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:07:57:086.927 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:07:57:087.046 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553277086706 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:07:57:087.074 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:07:59:001.749 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:07:59:001.911 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:07:59:001.955 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:07:59:001.986 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:07:59:002.334 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:07:59:002.372 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:07:59:002.387 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:07:59:002.401 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:07:59:003.759 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:07:59:004.967 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:07:59:044.943 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:07:59:045.077 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:07:59:045.135 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:07:59:045.180 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:07:59:045.223 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:07:59:045.368 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:07:59:045.400 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:07:59:045.415 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:07:59:045.428 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:07:59:045.440 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:07:59:193.349 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:07:59:193.505 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:07:59:193.580 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:07:59:193.634 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:07:59:193.648 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:07:59:193.676 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:07:59:193.705 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:07:59:193.738 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:07:59:193.752 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:07:59:193.763 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:01:916.808 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-06-04 Thu 06:08:01:916.867 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:01:916.913 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: SANDESH: Send FAILED: 1780553281916677 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 129 dequeues = 129 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 164 dequeues = 164 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553277086663 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553277086926 ] max_sm_queue_count = 6 ] 2026-06-04 Thu 06:08:01:916.926 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:01:916.934 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:01:917.024 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: SANDESH: Send FAILED: 1780553281916783 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:01:917.045 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638531708480, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:04:002.417 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:04:002.573 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:04:004.616 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:04:004.653 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:04:004.695 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:04:004.999 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:04:005.053 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:04:005.073 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:04:005.089 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:04:005.469 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:04:045.462 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:04:050.568 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:04:050.670 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:04:052.810 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:04:052.864 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:04:052.884 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:04:052.934 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:04:053.053 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:04:053.124 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:04:053.150 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:04:193.594 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:04:193.864 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:04:193.906 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:04:193.955 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:04:193.994 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:04:194.048 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:04:194.093 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:04:194.132 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:04:194.147 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:04:194.160 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:09:004.700 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:09:005.367 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:09:005.407 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:09:005.452 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:09:005.482 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:09:005.520 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:09:005.769 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:09:005.822 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:09:005.848 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:09:005.865 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:09:052.903 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:09:053.077 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:09:053.147 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:09:053.183 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:09:053.211 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:09:053.467 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:09:053.500 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:09:053.514 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:09:053.536 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:09:053.549 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:09:193.942 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:09:194.155 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:09:194.192 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:09:194.235 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:09:194.247 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:09:194.273 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:09:194.303 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:09:194.336 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:09:194.350 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:09:194.361 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:13:086.773 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:08:13:086.983 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-04 Thu 06:08:13:087.068 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-04 Thu 06:08:13:087.082 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:08:13:087.169 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553293086844 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:08:13:087.179 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:08:13:087.194 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:08:13:087.394 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: SANDESH: Send FAILED: 1780553293087069 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 68, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:13:087.417 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638548502080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:13:087.604 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1):EvTcpConnectFail 2026-06-04 Thu 06:08:13:087.811 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-04 Thu 06:08:13:087.830 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Connect : EvTcpConnectFail 2026-06-04 Thu 06:08:13:087.938 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:08:13:087.972 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Idle 2026-06-04 Thu 06:08:13:088.017 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:13:088.121 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SANDESH: Send FAILED: 1780553293087891 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 70, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:13:088.141 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:13:088.152 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:13:088.168 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SANDESH: Send FAILED: 1780553293087916 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:08:13:088.178 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:08:13:088.186 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:08:13:088.212 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:13:088.330 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: SANDESH: Send FAILED: 1780553293088008 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 70, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:13:088.346 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638130337344, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:14:005.474 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:14:005.837 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:14:005.890 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:14:005.920 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:14:005.948 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:14:006.114 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:14:006.193 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:14:006.208 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:14:006.220 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:14:006.234 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:14:053.554 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:14:053.705 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:14:053.745 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:14:053.779 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:14:053.817 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:14:054.052 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:14:054.092 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:14:054.113 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:14:054.130 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:14:054.143 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:14:194.234 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:14:194.453 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:14:194.493 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:14:194.548 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:14:194.595 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:14:194.637 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:14:194.701 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:14:194.719 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:14:194.732 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:14:194.744 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:19:006.193 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:19:006.376 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:19:006.415 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:19:006.444 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:19:006.475 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:19:006.833 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:19:006.868 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:19:006.884 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:19:006.896 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:19:006.908 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:19:054.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:19:054.333 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:19:054.370 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:19:054.397 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:19:054.429 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:19:054.734 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:19:054.767 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:19:054.791 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:19:054.808 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:19:054.826 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:19:194.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:19:194.761 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:19:194.818 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:19:194.865 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:19:194.914 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:19:194.933 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:19:194.947 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:19:194.983 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:19:195.023 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:19:195.041 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:24:006.952 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:24:007.188 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:24:007.248 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:24:007.310 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:24:007.360 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:24:007.654 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:24:007.703 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:24:007.719 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:24:007.735 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:24:007.749 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:24:054.848 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:24:055.051 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:24:055.111 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:24:055.156 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:24:055.195 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:24:055.434 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:24:055.492 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:24:055.538 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:24:055.553 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:24:055.565 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:24:194.868 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:24:195.063 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:24:195.090 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:24:195.157 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:24:195.199 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:24:195.260 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:24:195.280 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:24:195.355 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:24:195.369 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:24:195.412 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:29:007.749 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:29:007.906 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:29:007.958 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:29:007.995 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:29:008.032 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:29:008.280 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:29:008.323 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:29:008.337 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:29:008.350 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:29:008.370 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:29:055.507 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:29:055.697 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:29:055.738 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:29:055.781 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:29:055.807 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:29:055.984 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:29:056.053 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:29:056.070 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:29:056.086 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:29:056.112 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:29:195.161 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:29:195.409 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:29:195.479 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:29:195.514 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:29:195.550 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:29:195.664 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:29:195.736 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:29:195.753 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:29:195.767 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:29:195.782 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:31:917.205 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:31:917.376 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553311917018 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 136 dequeues = 136 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 174 dequeues = 174 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553293087972 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553293088211 ] max_sm_queue_count = 6 ] 2026-06-04 Thu 06:08:31:917.415 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:31:917.430 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:31:917.694 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553311917163 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:31:917.719 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:34:008.382 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:34:008.575 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:34:008.619 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:34:008.655 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:34:008.712 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:34:009.038 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:34:009.066 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:34:009.081 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:34:009.093 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:34:009.122 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:34:056.079 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:34:056.269 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:34:056.314 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:34:056.353 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:34:056.390 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:34:056.542 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:34:056.580 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:34:056.603 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:34:056.621 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:34:056.640 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:34:195.763 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:34:195.899 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:34:195.921 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:34:195.953 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:34:195.982 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:34:196.009 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:34:196.040 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:34:196.054 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:34:196.066 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:34:196.077 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:36:906.413 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:36:906.518 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: SANDESH: Send FAILED: 1780553316906350 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.384009 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-06-04 Thu 06:08:36:906.533 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-06-04 Thu 06:08:39:009.122 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:39:009.277 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:39:009.318 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:39:009.347 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:39:009.374 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:39:009.817 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:39:009.880 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:39:009.902 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:39:009.921 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:39:009.939 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:39:056.627 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:39:056.736 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:39:056.775 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:39:056.805 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:39:056.833 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:39:057.052 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:39:057.077 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:39:057.089 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:39:057.114 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:39:057.136 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:39:196.020 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:39:196.371 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:39:196.412 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:39:196.458 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:39:196.511 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:39:196.545 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:39:196.582 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:39:196.599 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:39:196.613 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:39:196.629 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:43:088.111 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:08:43:088.338 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-04 Thu 06:08:43:088.441 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:08:43:088.466 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553323088193 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:08:43:088.477 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:08:43:088.486 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:08:43:088.580 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SANDESH: Send FAILED: 1780553323088426 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 76, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:43:088.608 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:43:088.631 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-04 Thu 06:08:43:088.654 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1):EvTcpConnectFail 2026-06-04 Thu 06:08:43:088.686 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-04 Thu 06:08:43:088.696 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Connect : EvTcpConnectFail 2026-06-04 Thu 06:08:43:088.787 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:08:43:088.812 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Idle 2026-06-04 Thu 06:08:43:088.859 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:43:088.953 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: SANDESH: Send FAILED: 1780553323088744 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:43:088.970 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:43:088.978 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:43:088.994 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: SANDESH: Send FAILED: 1780553323088768 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:08:43:089.000 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:08:43:089.007 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:08:43:089.031 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:08:43:089.125 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: SANDESH: Send FAILED: 1780553323088849 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:08:43:089.144 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638168114752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:08:44:009.931 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:44:010.082 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:44:010.150 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:44:010.192 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:44:010.230 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:44:011.049 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:44:011.094 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:44:011.126 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:44:011.142 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:44:011.155 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:44:057.135 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:44:057.265 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:44:057.315 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:44:057.341 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:44:057.370 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:44:057.679 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:44:057.714 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:44:057.729 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:44:057.742 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:44:057.755 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:44:196.449 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:44:196.608 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:44:196.642 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:44:196.677 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:44:196.708 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:44:196.744 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:44:196.759 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:44:196.771 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:44:196.781 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:44:196.810 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:49:011.217 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:49:011.378 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:49:011.427 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:49:011.457 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:49:011.485 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:49:011.808 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:49:011.843 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:49:011.858 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:49:011.882 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:49:011.897 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:49:059.398 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:49:059.534 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:49:059.569 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:49:059.597 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:49:059.625 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:49:063.656 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:49:063.721 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:49:063.738 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:49:063.754 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:49:063.767 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:49:196.675 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:49:196.837 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:49:196.862 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:49:196.894 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:49:196.924 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:49:196.952 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:49:196.984 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:49:196.998 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:49:197.010 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:49:197.021 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:54:011.903 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:54:012.059 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:54:012.126 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:54:012.172 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:54:012.206 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:54:012.416 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:54:012.463 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:54:012.483 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:54:012.509 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:54:012.527 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:54:063.753 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:54:063.894 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:54:063.932 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:54:063.962 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:54:063.999 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:54:064.193 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:54:064.224 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:54:064.243 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:54:064.279 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:54:064.297 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:54:197.274 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:54:197.387 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:54:197.424 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:54:197.454 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:54:197.485 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:54:197.525 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:54:197.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:54:197.558 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:54:197.570 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:54:197.589 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:08:59:013.826 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:08:59:013.984 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:08:59:014.031 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:08:59:014.063 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:08:59:014.125 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:08:59:014.448 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:08:59:014.481 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:08:59:014.495 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:08:59:014.508 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:08:59:014.521 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:08:59:064.261 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:08:59:064.401 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:08:59:064.435 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:08:59:064.463 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:08:59:064.490 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:08:59:064.747 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:08:59:064.782 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:08:59:064.797 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:08:59:064.823 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:08:59:064.853 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:08:59:197.588 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:08:59:197.734 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:08:59:197.785 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:08:59:197.809 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:08:59:197.819 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:08:59:197.860 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:08:59:197.887 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:08:59:197.919 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:08:59:197.934 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:08:59:197.945 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:09:01:917.480 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-06-04 Thu 06:09:01:917.529 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:01:917.580 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553341917333 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 144 dequeues = 144 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 185 dequeues = 185 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553323088811 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553323089030 ] max_sm_queue_count = 6 ] 2026-06-04 Thu 06:09:01:917.592 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:01:917.600 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:01:917.702 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553341917455 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 81, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:01:917.734 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:04:014.990 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:09:04:015.181 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:09:04:015.226 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:09:04:015.264 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:09:04:015.360 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:09:04:015.578 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298e290 2026-06-04 Thu 06:09:04:015.638 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990410 2026-06-04 Thu 06:09:04:015.730 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298ee50 2026-06-04 Thu 06:09:04:015.750 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298d770 2026-06-04 Thu 06:09:04:015.766 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298f930 2026-06-04 Thu 06:09:04:064.811 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:09:04:064.940 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:09:04:064.976 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:09:04:065.013 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:09:04:065.040 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:09:04:065.222 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2993a70 2026-06-04 Thu 06:09:04:065.258 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29924b0 2026-06-04 Thu 06:09:04:065.292 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2992f90 2026-06-04 Thu 06:09:04:065.307 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x29919d0 2026-06-04 Thu 06:09:04:065.319 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2990ef0 2026-06-04 Thu 06:09:04:197.850 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:09:04:198.013 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:09:04:198.069 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298cc50 2026-06-04 Thu 06:09:04:198.096 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298c130 2026-06-04 Thu 06:09:04:198.118 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:09:04:198.150 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:09:04:198.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:09:04:198.210 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b830 2026-06-04 Thu 06:09:04:198.224 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x298b180 2026-06-04 Thu 06:09:04:198.235 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: -1 0x2986180 2026-06-04 Thu 06:09:09:015.663 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298e290 2026-06-04 Thu 06:09:09:015.818 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990410 2026-06-04 Thu 06:09:09:015.864 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298ee50 2026-06-04 Thu 06:09:09:015.907 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298d770 2026-06-04 Thu 06:09:09:015.935 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298f930 2026-06-04 Thu 06:09:09:017.282 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298e290 2026-06-04 Thu 06:09:09:017.311 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:017.338 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:017.355 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2990410 2026-06-04 Thu 06:09:09:017.362 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:017.388 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:017.400 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298d770 2026-06-04 Thu 06:09:09:017.406 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:017.416 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:017.427 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298f930 2026-06-04 Thu 06:09:09:017.433 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:017.443 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:017.454 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298ee50 2026-06-04 Thu 06:09:09:017.460 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:017.471 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:017.980 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:018.004 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:018.028 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:018.042 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:018.049 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:018.060 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:018.068 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:018.078 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:018.086 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:018.097 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:018.125 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:018.208 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:018.229 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:018.244 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:018.256 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:018.230 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:09:021.325 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553349018162 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:09:021.336 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:09:09:065.292 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2993a70 2026-06-04 Thu 06:09:09:065.455 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29924b0 2026-06-04 Thu 06:09:09:065.503 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2992f90 2026-06-04 Thu 06:09:09:065.542 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x29919d0 2026-06-04 Thu 06:09:09:065.580 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2990ef0 2026-06-04 Thu 06:09:09:068.138 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2993a70 2026-06-04 Thu 06:09:09:068.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:068.206 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:068.224 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x29924b0 2026-06-04 Thu 06:09:09:068.246 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:068.257 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:068.270 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2992f90 2026-06-04 Thu 06:09:09:068.276 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:068.286 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:068.297 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x29919d0 2026-06-04 Thu 06:09:09:068.303 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:068.313 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:068.325 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2990ef0 2026-06-04 Thu 06:09:09:068.332 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:068.341 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:068.772 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:068.796 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:068.814 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:068.826 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:068.842 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:068.853 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:068.869 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:068.881 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:068.896 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:068.914 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:068.945 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:069.789 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:09:069.833 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553349068984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:09:069.844 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:09:09:069.976 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:069.999 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:070.020 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:070.032 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:199.513 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298cc50 2026-06-04 Thu 06:09:09:199.707 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298c130 2026-06-04 Thu 06:09:09:199.769 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b830 2026-06-04 Thu 06:09:09:199.808 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x298b180 2026-06-04 Thu 06:09:09:199.862 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x2986180 2026-06-04 Thu 06:09:09:199.924 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298cc50 2026-06-04 Thu 06:09:09:199.935 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:199.965 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:199.981 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298c130 2026-06-04 Thu 06:09:09:200.001 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:200.014 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:200.026 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298b830 2026-06-04 Thu 06:09:09:200.033 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:200.044 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:200.055 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x298b180 2026-06-04 Thu 06:09:09:200.062 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:200.072 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:200.082 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: RAC_Connect status: 0 0x2986180 2026-06-04 Thu 06:09:09:200.087 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:09:09:200.093 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-06-04 Thu 06:09:09:211.405 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:211.455 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:211.481 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:211.492 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:211.509 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:211.516 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:211.529 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:211.536 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:211.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-06-04 Thu 06:09:09:211.565 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-06-04 Thu 06:09:09:211.586 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:214.245 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:09:214.291 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553349211613 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:09:214.300 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:09:09:216.524 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:216.564 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:216.575 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:09:216.584 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-06-04 Thu 06:09:13:094.680 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:09:13:095.878 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-04 Thu 06:09:13:096.035 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:09:13:096.072 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553353094763 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:13:096.100 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:09:13:096.123 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:09:13:096.290 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553353096007 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 235, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 86, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:13:110.067 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:13:110.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-04 Thu 06:09:13:110.190 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: -(-1):EvTcpConnectFail 2026-06-04 Thu 06:09:13:113.793 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-06-04 Thu 06:09:13:113.835 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Connect : EvTcpConnectFail 2026-06-04 Thu 06:09:13:114.187 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-04 Thu 06:09:13:114.234 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Idle 2026-06-04 Thu 06:09:13:114.315 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:13:114.483 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553353114073 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 235, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:13:114.502 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:13:114.514 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:13:114.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553353114131 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:13:114.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:09:13:114.573 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:09:13:114.585 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:13:114.716 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553353114300 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:13:114.733 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:31:920.352 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:31:920.441 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553371920217 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 154 dequeues = 154 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 198 dequeues = 198 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553353114233 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553353114584 ] max_sm_queue_count = 6 ] 2026-06-04 Thu 06:09:31:920.467 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:31:920.475 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:31:920.566 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: SANDESH: Send FAILED: 1780553371920320 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 91, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:31:920.579 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638527510080, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:36:909.735 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:09:36:909.825 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553376909699 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.384009 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-06-04 Thu 06:09:36:909.852 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-06-04 Thu 06:09:43:119.383 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-04 Thu 06:09:43:119.618 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-04 Thu 06:09:43:119.714 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:09:43:119.740 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553383119453 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:43:119.749 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:09:43:119.759 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:09:43:119.941 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SANDESH: Send FAILED: 1780553383119700 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 94, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:43:119.964 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:09:43:126.855 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:52224::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-04 Thu 06:09:43:126.886 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:52224::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:09:43:126.896 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638555644416, Pid 1]: 10.0.0.38:52224::10.0.0.38:8086(-1):EvTcpConnected 2026-06-04 Thu 06:09:43:132.743 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:09:43:132.783 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:09:43:132.825 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: ClientInit 2026-06-04 Thu 06:09:43:132.838 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 7 2026-06-04 Thu 06:09:43:132.925 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:09:43:132.934 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:09:43:132.953 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: SANDESH: Send FAILED: 1780553383132816 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:09:43:132.960 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:09:43:132.967 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:09:43:132.973 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:09:43:133.002 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:09:43:133.008 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:09:43:133.118 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: SANDESH: Send FAILED: 1780553383132912 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 237, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 96, _iter59->first = sent _iter59->second = 63, ] ] ] 2026-06-04 Thu 06:09:43:133.134 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638126138944, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:09:43:208.747 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: OnMessage control in state: ClientInit 2026-06-04 Thu 06:09:43:208.814 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-04 Thu 06:09:43:208.825 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-04 Thu 06:09:43:208.846 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Received Ctrl Message with size 0 2026-06-04 Thu 06:09:43:209.015 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 483087 calls = 59 average_bytes = 8187 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 154 dequeues = 154 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 198 dequeues = 198 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780553353114233 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780553353114584 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 17 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 130844 calls = 14 average_bytes = 9346 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 34 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 237, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 96, _iter59->first = sent _iter59->second = 63, ] ] max_sm_queue_count = 6 proxy seq 89 2026-06-04 Thu 06:09:43:209.058 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:09:43:209.118 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 51 2026-06-04 Thu 06:09:43:209.139 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:09:43:209.166 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.384009 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 11 2026-06-04 Thu 06:09:43:209.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:09:43:209.184 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-04 Thu 06:09:43:209.228 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Established 2026-06-04 Thu 06:09:46:986.747 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: OnMessage control in state: Established 2026-06-04 Thu 06:09:46:986.851 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-06-04 Thu 06:09:46:986.863 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Established : EvSandeshMessageRecv 2026-06-04 Thu 06:09:46:986.918 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: Received Ctrl Message with size 3 2026-06-04 Thu 06:09:46:986.932 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 90, total = 0 2026-06-04 Thu 06:09:46:986.944 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 53, total = 0 2026-06-04 Thu 06:09:46:986.954 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 11, total = 0 2026-06-04 Thu 06:09:46:986.961 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-04 Thu 06:10:01:920.858 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638138734144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-06-04 Thu 06:11:01:921.617 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638134535744, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-06-04 Thu 06:12:01:924.694 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638142932544, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-06-04 Thu 06:12:09:355.184 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: Processing scm::EvStop in state Established 2026-06-04 Thu 06:12:09:356.106 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: Idle 2026-06-04 Thu 06:12:09:356.236 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-04 Thu 06:12:09:356.249 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-06-04 Thu 06:12:09:356.375 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: SANDESH: Send FAILED: 1780553529356220 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1780552716530089 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 5912 calls = 10 average_bytes = 591 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 656188 calls = 75 average_bytes = 8749 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 217 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 46 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 237, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 98, _iter59->first = sent _iter59->second = 82, ] ] ] 2026-06-04 Thu 06:12:09:356.397 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 139638163916352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:12:36:554.695 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: http-server-port 8091 2026-06-04 Thu 06:12:36:554.772 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Max-tasks 16 2026-06-04 Thu 06:12:36:554.779 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Max-slice 100 2026-06-04 Thu 06:12:36:554.785 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Collectors 10.0.0.38:8086 2026-06-04 Thu 06:12:36:554.799 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Collectors 10.0.0.50:8086 2026-06-04 Thu 06:12:36:554.803 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Collectors 10.0.0.33:8086 2026-06-04 Thu 06:12:36:554.824 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: ROLE : Generator 2026-06-04 Thu 06:12:36:554.830 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-06-04 Thu 06:12:36:554.834 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5888-1. 2026-06-04 Thu 06:12:36:554.839 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: NODE TYPE : Database 2026-06-04 Thu 06:12:36:554.844 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: INSTANCE ID : 0 2026-06-04 Thu 06:12:36:554.848 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-06-04 Thu 06:12:36:554.854 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-04 Thu 06:12:36:623.278 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: HTTP Introspect Init 2026-06-04 Thu 06:12:36:623.436 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-06-04 Thu 06:12:36:623.525 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Sandesh Http Server Port: 8091 2026-06-04 Thu 06:12:36:623.590 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-06-04 Thu 06:12:36:623.600 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-04 Thu 06:12:36:623.932 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Idle 2026-06-04 Thu 06:12:36:624.082 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: No Client: 1780553556624024 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780553556623943 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 06:12:36:624.248 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-04 Thu 06:12:36:624.264 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-04 Thu 06:12:36:624.275 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-06-04 Thu 06:12:36:624.975 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvStart in state Idle 2026-06-04 Thu 06:12:36:625.056 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Disconnect 2026-06-04 Thu 06:12:36:625.130 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-04 Thu 06:12:36:625.360 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-04 Thu 06:12:36:625.431 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:12:36:625.450 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: SANDESH: Send FAILED: 1780553556625049 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-04 Thu 06:12:36:625.458 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:12:36:625.467 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:12:36:625.548 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: SANDESH: Send FAILED: 1780553556625115 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780553556623943 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 06:12:36:625.566 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:12:36:625.574 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:12:36:626.246 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: SANDESH: Send FAILED: 1780553556625188 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780553556623943 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 06:12:36:626.279 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:12:36:626.293 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:12:36:626.308 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: SANDESH: Send FAILED: 1780553556625207 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-06-04 Thu 06:12:36:626.314 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:12:36:626.321 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:12:36:626.366 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: SANDESH: Send FAILED: 1780553556625415 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1780553556623943 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 06:12:36:626.378 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-04 Thu 06:12:36:663.364 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-06-04 Thu 06:12:51:686.601 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193459996224, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-06-04 Thu 06:12:51:686.685 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193459996224, Pid 1]: SANDESH: Send FAILED: 1780553571686494 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-04 Thu 06:12:51:686.695 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193459996224, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-04 Thu 06:12:51:688.303 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:44642::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-06-04 Thu 06:12:51:688.340 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: 10.0.0.38:44642::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-04 Thu 06:12:51:688.351 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: 10.0.0.38:44642::10.0.0.50:8086(-1):EvTcpConnected 2026-06-04 Thu 06:12:51:688.398 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a4b540 2026-06-04 Thu 06:12:51:688.406 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.419 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.434 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a50540 2026-06-04 Thu 06:12:51:688.439 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.461 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.472 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a50e30 2026-06-04 Thu 06:12:51:688.477 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.484 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.492 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a51af0 2026-06-04 Thu 06:12:51:688.497 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.503 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.513 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a527b0 2026-06-04 Thu 06:12:51:688.517 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.524 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.532 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a53470 2026-06-04 Thu 06:12:51:688.537 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.543 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.552 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a54030 2026-06-04 Thu 06:12:51:688.557 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.563 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.572 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a55870 2026-06-04 Thu 06:12:51:688.577 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.587 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.600 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a57110 2026-06-04 Thu 06:12:51:688.606 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.616 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.628 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a54c70 2026-06-04 Thu 06:12:51:688.640 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.649 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.658 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a564d0 2026-06-04 Thu 06:12:51:688.663 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.669 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.679 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a57d50 2026-06-04 Thu 06:12:51:688.683 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.690 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.700 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a58990 2026-06-04 Thu 06:12:51:688.704 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.710 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.719 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a59590 2026-06-04 Thu 06:12:51:688.723 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.730 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.739 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: RAC_Connect status: 0 0x1a5a1f0 2026-06-04 Thu 06:12:51:688.744 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: Connected to REDIS... 2026-06-04 Thu 06:12:51:688.750 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-06-04 Thu 06:12:51:688.439 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-06-04 Thu 06:12:51:688.978 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-04 Thu 06:12:51:689.039 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: ClientInit 2026-06-04 Thu 06:12:51:689.055 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5888-1.:contrail-query-engine:0:Database count 1 2026-06-04 Thu 06:12:51:689.152 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:689.162 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:12:51:689.179 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: SANDESH: Send FAILED: 1780553571689033 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-06-04 Thu 06:12:51:689.192 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:12:51:689.199 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:689.205 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-04 Thu 06:12:51:689.223 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:689.229 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-04 Thu 06:12:51:689.339 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: SANDESH: Send FAILED: 1780553571689127 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780553556623943 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-04 Thu 06:12:51:689.356 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-04 Thu 06:12:51:689.764 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:689.783 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:689.797 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:689.804 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:689.817 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:689.828 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:689.842 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:689.860 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:689.875 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:689.885 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:689.948 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:689.972 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:689.983 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:689.994 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.005 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.046 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:690.055 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:12:51:690.072 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: SANDESH: Send FAILED: 1780553571689937 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-06-04 Thu 06:12:51:690.079 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:12:51:690.086 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:690.092 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:12:51:690.103 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: SANDESH: Send FAILED: 1780553571689967 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:12:51:690.114 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:12:51:690.163 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.171 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.183 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.190 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.200 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.210 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.226 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.236 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.249 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.256 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.266 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.273 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.287 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.297 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.316 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.328 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.343 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.353 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.375 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.405 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.419 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.429 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.439 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.453 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.469 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.479 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.488 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.505 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-06-04 Thu 06:12:51:690.515 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-06-04 Thu 06:12:51:690.524 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:690.548 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-06-04 Thu 06:12:51:690.552 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:12:51:691.407 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SANDESH: Send FAILED: 1780553571690396 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:12:51:691.437 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:12:51:691.449 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-06-04 Thu 06:12:51:691.457 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-04 Thu 06:12:51:691.471 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SANDESH: Send FAILED: 1780553571690465 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-04 Thu 06:12:51:691.477 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-06-04 Thu 06:12:51:691.953 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: OnMessage control in state: ClientInit 2026-06-04 Thu 06:12:51:691.995 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-04 Thu 06:12:51:692.003 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-06-04 Thu 06:12:51:692.019 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Received Ctrl Message with size 0 2026-06-04 Thu 06:12:51:692.132 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1780553556623943 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.50:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2026-06-04 Thu 06:12:51:692.174 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:12:51:692.193 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5888-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 8 2026-06-04 Thu 06:12:51:692.222 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-04 Thu 06:12:51:692.229 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-06-04 Thu 06:12:51:692.235 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-06-04 Thu 06:12:51:692.260 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193491306048, Pid 1]: Established 2026-06-04 Thu 06:13:36:624.798 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-06-04 Thu 06:14:36:625.399 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193575994944, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-06-04 Thu 06:15:36:631.888 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-06-04 Thu 06:16:36:633.255 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-06-04 Thu 06:17:36:634.680 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193459996224, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-06-04 Thu 06:18:36:635.359 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-06-04 Thu 06:19:36:636.025 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193487107648, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-06-04 Thu 06:20:36:636.588 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-06-04 Thu 06:21:36:637.177 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-06-04 Thu 06:22:36:638.444 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193487107648, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-06-04 Thu 06:23:36:639.059 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-06-04 Thu 06:24:36:640.079 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193168356928, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-06-04 Thu 06:25:36:640.637 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193588590144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-06-04 Thu 06:26:36:641.411 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193487107648, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-06-04 Thu 06:27:36:641.990 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 15 without seqno , total = 0 2026-06-04 Thu 06:28:36:642.597 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193588590144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 16 without seqno , total = 0 2026-06-04 Thu 06:29:36:643.366 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193168356928, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 17 without seqno , total = 0 2026-06-04 Thu 06:29:52:252.318 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:53402 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-04 Thu 06:30:00:165.102 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:44280 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-04 Thu 06:30:36:644.657 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193168356928, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 18 without seqno , total = 0 2026-06-04 Thu 06:31:26:939.647 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:57754 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-04 Thu 06:31:36:645.920 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 19 without seqno , total = 0 2026-06-04 Thu 06:31:49:684.648 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-06-04 Thu 06:31:49:685.003 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID 113de638-5fdf-11f1-bd1b-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-06-04 Thu 06:31:49:685.070 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 :0:json query is: { end_time: 1780554709610736 , enqueue_time: 1780554709684190 , query_metadata: {"enqueue_time": 1780554709684032} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1780554691581498 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-04 Thu 06:31:49:685.219 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-06-04 Thu 06:31:49:685.235 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-06-04 Thu 06:31:49:685.304 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for 113de638-5fdf-11f1-bd1b-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-06-04 Thu 06:31:49:685.411 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 :0:json query is: { end_time: 1780554709610736 , enqueue_time: 1780554709684190 , query_metadata: {"enqueue_time": 1780554709684032} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1780554691581498 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-04 Thu 06:31:49:817.006 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:817.114 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:817.129 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:817.156 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:817.167 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:817.189 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:817.200 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:817.221 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:817.231 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:817.252 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:817.349 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 :0:json query is: { end_time: 1780554709610736 , enqueue_time: 1780554709684190 , query_metadata: {"enqueue_time": 1780554709684032} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1780554691581498 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-04 Thu 06:31:49:817.554 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:817.667 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:817.770 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:817.879 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:817.978 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.083 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.175 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.279 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.396 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.489 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.583 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.673 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.764 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.866 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:818.957 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.050 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.139 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.232 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.352 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.441 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.535 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.625 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.715 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.808 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:819.914 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.006 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.100 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.188 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.283 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.377 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.471 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.635 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:820.732 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:822.178 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 11 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:822.227 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:822.243 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 11 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:822.272 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:822.285 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 11 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:822.303 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:822.314 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 11 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:822.329 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:822.340 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 11 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:822.366 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 11 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:822.654 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:822.820 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:822.980 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:823.130 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:823.285 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:823.416 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:823.563 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:823.715 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:823.874 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:824.032 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:824.191 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193499702848, Pid 1]: [SYS_INFO]: QEQueryLog: 113de638-5fdf-11f1-bd1b-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:49:825.074 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:825.098 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:825.106 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:825.114 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:825.121 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:825.128 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:825.141 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:825.149 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:825.156 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:49:825.163 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: 113de638-5fdf-11f1-bd1b-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:49:825.627 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 44 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-06-04 Thu 06:31:49:826.388 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 44 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-06-04 Thu 06:31:49:826.708 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":6, "count":44} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-06-04 Thu 06:31:49:826.931 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193495504448, Pid 1]: [SYS_INFO]: QELog: Finished: QID 113de638-5fdf-11f1-bd1b-00000a000026 Table ObjectVNTable Time(ms) 141 RedisTime(ms) 0 MergeTime(ms) 0 Rows 44 EnQ-delay0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-06-04 Thu 06:31:49:826.962 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193592788544, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5888-1. table = ObjectVNTable query_stats= [ qid = 113de638-5fdf-11f1-bd1b-00000a000026 where = [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-98789151:ctest-vn22-29431496", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 131,0, 139,0, 136,0, chunk_select_time = 0,3, 0,0, 0,1, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 141 rows = 44 enq_delay = 0 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-06-04 Thu 06:31:49:827.074 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193495504448, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-06-04 Thu 06:31:52:835.705 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-06-04 Thu 06:31:52:836.110 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID 131eb725-5fdf-11f1-8c01-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-06-04 Thu 06:31:52:836.148 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 :0:json query is: { end_time: 1780554712821914 , enqueue_time: 1780554712835033 , query_metadata: {"enqueue_time": 1780554712834912} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1780554691581498 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "a1a7de84-ba54-4413-8fc4-daf6792cce3d", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-04 Thu 06:31:52:836.279 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-06-04 Thu 06:31:52:836.293 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-06-04 Thu 06:31:52:836.343 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for 131eb725-5fdf-11f1-8c01-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-06-04 Thu 06:31:52:836.546 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 :0:json query is: { end_time: 1780554712821914 , enqueue_time: 1780554712835033 , query_metadata: {"enqueue_time": 1780554712834912} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1780554691581498 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "a1a7de84-ba54-4413-8fc4-daf6792cce3d", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-04 Thu 06:31:52:889.316 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.364 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.372 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.381 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.388 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.396 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.402 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.408 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.415 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 1 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.421 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 1 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.461 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.468 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.474 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.495 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.501 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.507 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.513 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.519 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.525 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.531 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.549 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.556 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.562 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.568 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.574 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.580 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.586 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 0 and 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.599 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.610 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 UNION between tables of sizes 5 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-06-04 Thu 06:31:52:889.620 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: 131eb725-5fdf-11f1-8c01-00000a000026 Resulting size of set 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-06-04 Thu 06:31:52:889.668 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 :0:json query is: { end_time: 1780554712821914 , enqueue_time: 1780554712835033 , query_metadata: {"enqueue_time": 1780554712834912} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1780554691581498 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "a1a7de84-ba54-4413-8fc4-daf6792cce3d", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-06-04 Thu 06:31:52:889.951 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 ObjectVMTable :2:Column ObjectTypeName1 Table ObjectVMTable object_id a1a7de84-ba54-4413-8fc4-daf6792cce3d src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:52:890.178 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id a1a7de84-ba54-4413-8fc4-daf6792cce3d src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:52:890.281 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id a1a7de84-ba54-4413-8fc4-daf6792cce3d src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:52:890.380 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id a1a7de84-ba54-4413-8fc4-daf6792cce3d src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:52:890.474 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id a1a7de84-ba54-4413-8fc4-daf6792cce3d src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:52:890.582 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QEQueryLog: 131eb725-5fdf-11f1-8c01-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id a1a7de84-ba54-4413-8fc4-daf6792cce3d src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-06-04 Thu 06:31:52:890.721 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 6 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-06-04 Thu 06:31:52:890.807 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 6 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-06-04 Thu 06:31:52:890.937 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":1, "count":6} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-06-04 Thu 06:31:52:891.429 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: Finished: QID 131eb725-5fdf-11f1-8c01-00000a000026 Table ObjectVMTable Time(ms) 55 RedisTime(ms) 0 MergeTime(ms) 0 Rows 6 EnQ-delay1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-06-04 Thu 06:31:52:891.521 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-06-04 Thu 06:31:52:891.435 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193478710848, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5888-1. table = ObjectVMTable query_stats= [ qid = 131eb725-5fdf-11f1-8c01-00000a000026 where = [[{"name": "ObjectId", "value": "a1a7de84-ba54-4413-8fc4-daf6792cce3d", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 52,0, 52,0, 52,0, chunk_select_time = 0,0, 0,0, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 55 rows = 6 enq_delay = 1 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-06-04 Thu 06:32:36:646.697 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193495504448, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 20 without seqno , total = 0 2026-06-04 Thu 06:33:36:647.470 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193588590144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 21 without seqno , total = 0 2026-06-04 Thu 06:34:36:650.120 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 22 without seqno , total = 0 2026-06-04 Thu 06:35:36:651.920 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 23 without seqno , total = 0 2026-06-04 Thu 06:36:36:652.611 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 24 without seqno , total = 0 2026-06-04 Thu 06:37:36:653.304 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193588590144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 25 without seqno , total = 0 2026-06-04 Thu 06:38:36:654.225 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193580193344, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 26 without seqno , total = 0 2026-06-04 Thu 06:39:36:655.030 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193588590144, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 27 without seqno , total = 0 2026-06-04 Thu 06:40:36:656.366 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 28 without seqno , total = 0 2026-06-04 Thu 06:41:36:657.076 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 29 without seqno , total = 0 2026-06-04 Thu 06:42:36:658.225 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193464194624, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 30 without seqno , total = 0 2026-06-04 Thu 06:43:36:658.910 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193164158528, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 31 without seqno , total = 0 2026-06-04 Thu 06:43:41:384.404 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:54374 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-06-04 Thu 06:44:36:659.688 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193503901248, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 32 without seqno , total = 0 2026-06-04 Thu 06:45:23:699.350 UTC cn-jenkins-deploy-platform-ansible-os-5888-1 [Thread 140193599928448, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:39192 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256