09/14/2026 02:29:49.927 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:29:49.927 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:29:49.927 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:29:49.939 7f4c44227900 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 09/14/2026 02:29:49.940 7f4c44227900 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 09/14/2026 02:29:49.941 7f4c44227cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:29:49.942 7f4c44227cc0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/14/2026 02:29:49.943 7f4c44227cc0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:29:49.943 7f4c44227cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:29:49.945 7f4c44227cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:49.945 7f4c44227cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:49.945 7f4c44227cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:49.945 7f4c44227cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:49.946 7f4c44227cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:29:49.946 7f4c44227cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.159 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 34066390464200495506104401349864287034569069555307667901028570954514233077951, 'start': '1789352986000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 30373082838202928724160042418427994613388523363013538760780361921428346658920, 'start': '1789352986000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 68637554150183532420370749674579367721565682994539209777985922929374487491584, 'start': '1789352986000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/14/2026 02:29:50.255 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.256 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> >> 09/14/2026 02:29:50.256 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.256 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-3.', _node_type='Database', _instance_id='0', _timestamp=1789352990256666, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:50.257 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-3.', _node_type='Database', _instance_id='0', _timestamp=1789352990256962, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:50.258 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.258 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:50.259 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.259 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.264 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]} >> 09/14/2026 02:29:50.264 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.784 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:29:51.076 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/14/2026 02:29:51.077 7f4c439b0f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:51.078 7f4c439b0f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. database_usage = [ << disk_space_used_1k = 5872348 disk_space_available_1k = 75218136 analytics_db_size_1k = 4 >>, ] >> 09/14/2026 02:29:51.078 7f4c439b0f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:51.721 7f4c46a3c840 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/14/2026 02:29:51.722 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-3.', _node_type='Database', _instance_id='0', _timestamp=1789352991722145, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:51.725 7f4c439b0f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:51.726 7f4c439b0f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:51.726 7f4c439b0f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.534 7f4c46a3c840 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 09/14/2026 02:29:53.942 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:53.944 7f4c43624d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:29:53.945 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:53.945 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:53.945 7f4c43624d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:29:53.945 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:53.945 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:29:53.946 7f4c43624d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.944 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:57.946 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:29:57.946 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:57.946 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:57.946 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:29:57.946 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:57.947 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:29:57.947 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:29:57.947 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.947 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:29:57.947 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.947 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.948 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:29:57.948 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:58.862 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-3.', _node_type='Database', _instance_id='0', _timestamp=1789352998862381, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:58.862 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=5907952, partition_space_available_1k=75182532, percentage_partition_space_used=7, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1260120, cpu_share=0.0, mem_res=1252048, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7992, cpu_share=0.0, mem_res=6276, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55884, cpu_share=0.0, mem_res=46384, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4353432, free=6655016, buffers=177280, cached=5298708, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=1.41, fifteen_min_avg=0.55, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:29:58.863 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:58.863 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]} >> 09/14/2026 02:29:58.863 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:58.863 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:58.863 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 5907952 partition_space_available_1k = 75182532 percentage_partition_space_used = 7 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1260120 cpu_share = 0.0 mem_res = 1252048 >>contrail-query-engine : << mem_virt = 7992 cpu_share = 0.0 mem_res = 6276 >>contrail-database-nodemgr : << mem_virt = 55884 cpu_share = 0.0 mem_res = 46384 >> } system_mem_usage = << total = 16175632 used = 4353432 free = 6655016 buffers = 177280 cached = 5298708 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.23 five_min_avg = 1.41 fifteen_min_avg = 0.55 cpu_share = 0 node_type = database-node >> >> 09/14/2026 02:29:58.863 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.949 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:01.951 7f4c43877400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:01.951 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:01.951 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:01.951 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:01.952 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:01.952 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:01.952 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:01.952 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.953 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:01.953 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.953 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.953 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:01.953 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:05.952 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:05.954 7f4c43877400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:05.954 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:05.954 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:05.954 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:05.954 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:05.954 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:05.955 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:05.955 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:05.955 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:05.955 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:05.955 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:05.956 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:05.956 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.954 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:09.956 7f4c43877400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:09.956 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:09.956 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:09.957 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:09.957 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:09.957 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:09.957 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:09.957 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.958 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:09.958 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.958 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:09.958 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:09.958 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:13.958 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:13.959 7f4c43877400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:13.960 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:13.961 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:13.961 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:13.961 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:13.961 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:17.963 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:17.964 7f4c43877400 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:30:17.965 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:17.966 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:17.966 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:47.975 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:30:47.976 7f4c44296a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:30:47.976 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:30:47.976 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:30:47.976 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:30:47.976 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:47.976 7f4c44296a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:30:47.977 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:47.977 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:47.977 7f4c44296a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 09/14/2026 02:30:47.977 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:50.515 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:30:50.805 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:50.805 7f4c44296a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. database_usage = [ << disk_space_used_1k = 6824016 disk_space_available_1k = 74266468 analytics_db_size_1k = 772 >>, ] >> 09/14/2026 02:30:50.805 7f4c44296a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:51.528 7f4c46a3c840 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-3.', _node_type='Database', _instance_id='0', _timestamp=1789353051528108, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Down', description='ClientInit to Idle on EvConnectTimerExpired')]), flag_infos=None, description='Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:30:51.530 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:51.535 7f4c43877c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:30:51.535 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:51.977 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:51.988 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:51.988 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:51.989 7f4c43877c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:55.661 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-3.', _node_type='Database', _instance_id='0', _timestamp=1789353055661713, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:30:55.662 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 09/14/2026 02:30:55.666 7f4c43877ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:00.812 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6838844, partition_space_available_1k=74251640, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1297520, cpu_share=0.16, mem_res=1285560, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10856, cpu_share=0.0, mem_res=9064, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55912, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5134012, free=4944556, buffers=199024, cached=6207744, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.47, five_min_avg=2.23, fifteen_min_avg=0.88, cpu_share=16.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:31:00.813 7f4c43877ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:21.989 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:31:21.989 7f4c439b0ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:31:21.990 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:31:21.990 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:31:21.990 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:31:21.990 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:21.990 7f4c439b0ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:31:21.990 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:21.991 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:21.991 7f4c439b0ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 2 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:31:21.991 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:25.991 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:25.993 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:25.993 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:25.993 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:25.993 7f4c439b0ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:31:25.993 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:25.994 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:25.994 7f4c439b0ae0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:31:25.994 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:25.994 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:25.994 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:50.454 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:31:50.676 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:55.994 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:31:55.994 7f4c439b0540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:31:55.995 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:31:55.995 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:31:55.995 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:31:55.995 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.995 7f4c439b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:31:55.995 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.996 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:55.996 7f4c439b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:31:55.996 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:59.997 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:59.999 7f4c439b0540 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:32:00.000 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:00.000 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:00.000 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:00.179 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6845188, partition_space_available_1k=74245296, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1291396, cpu_share=0.14, mem_res=1279440, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11312, cpu_share=0.0, mem_res=9508, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55956, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5377312, free=4689084, buffers=200264, cached=6218680, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.6, five_min_avg=2.53, fifteen_min_avg=1.06, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:32:00.179 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:30.007 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:32:30.007 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:32:30.008 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:34.011 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:34.013 7f4c439b0cc0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:32:34.014 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:34.014 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:34.014 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:50.475 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:32:50.775 7f4c43877540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:59.147 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6851992, partition_space_available_1k=74238492, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1307224, cpu_share=0.12, mem_res=1295252, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11652, cpu_share=0.0, mem_res=9836, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55936, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5259068, free=4800204, buffers=201612, cached=6224432, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.34, five_min_avg=2.82, fifteen_min_avg=1.25, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:32:59.148 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:04.018 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:33:04.018 7f4c43877d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:33:04.019 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:08.021 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:08.022 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:08.022 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:08.022 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:08.022 7f4c439b0540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:33:08.022 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:08.022 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:08.023 7f4c439b0540 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:33:08.023 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:08.023 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:08.023 7f4c439b0540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:38.023 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:33:38.023 7f4c43877860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:33:38.024 7f4c43877860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.028 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:42.029 7f4c43877860 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:42.029 7f4c43877860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:42.029 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:42.030 7f4c43877860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:33:42.030 7f4c43877860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:42.030 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:42.030 7f4c43877860 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:33:42.030 7f4c43877860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:42.031 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:42.031 7f4c43877860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:50.818 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:33:51.080 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:59.934 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6859064, partition_space_available_1k=74231420, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1297408, cpu_share=0.1, mem_res=1285428, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12200, cpu_share=0.0, mem_res=10376, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55964, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5318756, free=4732340, buffers=202684, cached=6231544, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.02, five_min_avg=2.97, fifteen_min_avg=1.4, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:33:59.935 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:12.031 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:34:12.032 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:34:12.032 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:12.033 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 7 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:34:12.034 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:16.036 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:16.038 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:16.038 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:16.038 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:16.038 7f4c438774a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:34:16.038 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:16.039 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:16.039 7f4c438774a0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:34:16.039 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:16.040 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:16.040 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:46.045 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:34:46.045 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:34:46.045 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 8 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:34:46.046 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.047 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:50.048 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:50.048 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:50.048 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:50.049 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:34:50.049 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:50.049 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:50.051 7f4c439b00e0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:34:50.051 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:50.052 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:50.052 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:50.510 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:34:50.739 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:00.601 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6866796, partition_space_available_1k=74223688, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352460, cpu_share=0.1, mem_res=1340412, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12424, cpu_share=0.0, mem_res=10592, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55996, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5379428, free=4657084, buffers=203884, cached=6242232, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.73, five_min_avg=3.25, fifteen_min_avg=1.59, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:35:00.601 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:20.059 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:35:20.059 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:35:20.060 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:35:20.060 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:35:20.060 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:35:20.060 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:20.060 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:35:20.061 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:20.061 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:20.061 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 9 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:35:20.061 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:24.062 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:24.063 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:24.064 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:24.064 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:24.064 7f4c43877b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:35:24.064 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:24.064 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:24.065 7f4c43877b80 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:35:24.065 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:24.065 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:24.065 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:50.546 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:35:50.792 7f4c43877b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:54.064 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:35:54.064 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:35:54.065 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:35:54.065 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:35:54.065 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:35:54.065 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:54.065 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:35:54.065 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:54.066 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:54.066 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 10 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:35:54.066 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:58.066 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:58.068 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:58.068 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:58.068 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:58.069 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:35:58.069 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:58.069 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:58.069 7f4c439b0180 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:35:58.070 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:58.070 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:58.070 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:59.417 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6873748, partition_space_available_1k=74216736, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1324404, cpu_share=0.11, mem_res=1312088, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13036, cpu_share=0.0, mem_res=11192, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56040, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5263228, free=4764060, buffers=205004, cached=6249412, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.42, five_min_avg=3.34, fifteen_min_avg=1.73, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:35:59.418 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:28.072 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:36:28.073 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:36:28.073 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:36:28.073 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:36:28.073 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:36:28.073 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.074 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:36:28.074 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.074 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.074 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 11 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:36:28.074 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:32.075 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:32.076 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:32.077 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:32.077 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:32.077 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:36:32.077 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:32.077 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:32.077 7f4c439b0180 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:36:32.078 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:32.078 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:32.078 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:50.695 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:36:51.209 7f4c439b0ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:59.386 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6881420, partition_space_available_1k=74209064, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1306612, cpu_share=0.09, mem_res=1294300, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13336, cpu_share=0.0, mem_res=11484, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56024, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5188920, free=4832644, buffers=206028, cached=6254100, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.14, five_min_avg=3.49, fifteen_min_avg=1.88, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:36:59.386 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:02.079 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:37:02.079 7f4c438779a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:02.080 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:02.081 7f4c438779a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 12 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:37:02.081 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:06.084 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:06.085 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:06.086 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:06.086 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:06.086 7f4c438779a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:37:06.086 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:06.086 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:06.087 7f4c438779a0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:37:06.087 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:06.087 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:06.087 7f4c438779a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:36.092 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:37:36.093 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:37:36.094 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:37:36.094 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:37:36.095 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:37:36.095 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:36.095 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:37:36.096 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:36.096 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:36.096 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 13 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:37:36.097 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.095 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:40.097 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:40.097 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:40.097 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:40.097 7f4c43877e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:37:40.097 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:40.097 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:40.098 7f4c43877e00 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:37:40.101 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:40.101 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:40.102 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:50.580 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:37:50.905 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:59.479 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6888844, partition_space_available_1k=74201640, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1300440, cpu_share=0.09, mem_res=1288124, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13896, cpu_share=0.0, mem_res=12036, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56044, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5236572, free=4770808, buffers=207100, cached=6267220, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.54, five_min_avg=3.74, fifteen_min_avg=2.07, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:37:59.480 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:10.097 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:38:10.098 7f4c439b0220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:38:10.098 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:38:10.099 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 14 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:38:10.100 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:14.099 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:14.101 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:14.101 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:14.101 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:14.102 7f4c439b0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:38:14.102 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:14.102 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:14.102 7f4c439b0220 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:38:14.103 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:14.103 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:14.103 7f4c439b0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:44.103 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:38:44.104 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:38:44.104 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:38:44.104 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:38:44.104 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:38:44.104 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:44.105 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:38:44.105 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:44.105 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:44.105 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 15 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:38:44.105 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:48.107 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:48.109 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:38:48.110 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:48.110 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:48.110 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:50.595 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:38:51.111 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:00.221 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6904908, partition_space_available_1k=74185576, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1315388, cpu_share=0.1, mem_res=1303092, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14100, cpu_share=0.0, mem_res=12232, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56060, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5291236, free=4709156, buffers=208060, cached=6273256, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.0, five_min_avg=3.77, fifteen_min_avg=2.19, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:39:00.222 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:18.111 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:39:18.112 7f4c439b02c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:39:18.112 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:39:18.112 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:39:18.112 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:39:18.112 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.113 7f4c439b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:39:18.113 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.113 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.113 7f4c439b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 16 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:39:18.113 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:22.115 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:22.117 7f4c439b02c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:39:22.118 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:22.118 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:22.118 7f4c439b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:50.584 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:39:50.868 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:52.117 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:39:52.118 7f4c44296400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:39:52.118 7f4c44296400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:39:52.118 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:39:52.118 7f4c44296400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:39:52.118 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:52.119 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:39:52.119 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:52.119 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:52.119 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 17 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:39:52.119 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:56.119 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:56.121 7f4c43877400 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:39:56.122 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:56.122 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:56.122 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:58.492 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6912392, partition_space_available_1k=74178092, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353448, cpu_share=0.1, mem_res=1341132, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14588, cpu_share=0.0, mem_res=12712, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56104, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5225756, free=4766692, buffers=208972, cached=6280280, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.34, five_min_avg=3.6, fifteen_min_avg=2.22, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:39:58.492 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:26.122 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:40:26.122 7f4c44296400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 18 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:40:26.123 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:30.125 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:30.127 7f4c44296400 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:30.127 7f4c44296400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:30.128 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:30.128 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:40:30.128 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:30.129 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:30.129 7f4c44296400 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:40:30.130 7f4c44296400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:30.130 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:30.130 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:50.628 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:40:50.920 7f4c43877d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:00.128 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:41:00.128 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:41:00.129 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:41:00.129 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:41:00.129 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:41:00.129 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.130 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:41:00.130 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.130 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.130 7f4c439b0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 19 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:41:00.130 7f4c439b0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.174 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6920460, partition_space_available_1k=74170024, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327116, cpu_share=0.09, mem_res=1314584, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14924, cpu_share=0.0, mem_res=13036, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56096, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5205388, free=4777736, buffers=209896, cached=6288616, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.13, five_min_avg=3.42, fifteen_min_avg=2.24, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:41:00.174 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.175 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6920460 partition_space_available_1k = 74170024 percentage_partition_space_used = 9 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1327116 cpu_share = 0.09 mem_res = 1314584 >>contrail-query-engine : << mem_virt = 14924 cpu_share = 0.0 mem_res = 13036 >>contrail-database-nodemgr : << mem_virt = 56096 cpu_share = 0.01 mem_res = 46380 >> } system_mem_usage = << total = 16175632 used = 5205388 free = 4777736 buffers = 209896 cached = 6288616 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.13 five_min_avg = 3.42 fifteen_min_avg = 2.24 cpu_share = 13.5 node_type = database-node >> >> 09/14/2026 02:41:00.175 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:04.133 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:04.135 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:41:04.136 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:04.136 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:04.136 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:34.140 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:34.141 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:34.142 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 20 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:41:34.142 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:38.143 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:38.144 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:38.145 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:38.145 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:38.145 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:41:38.145 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:38.145 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:38.146 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:41:38.146 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:38.146 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:38.146 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:50.562 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:41:50.800 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:59.478 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6927564, partition_space_available_1k=74162920, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339644, cpu_share=0.1, mem_res=1327148, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15208, cpu_share=0.0, mem_res=13308, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56116, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5214392, free=4757504, buffers=210664, cached=6299056, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.24, five_min_avg=3.34, fifteen_min_avg=2.29, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:41:59.479 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:08.153 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:42:08.155 7f4c44296400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:42:08.157 7f4c44296400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:42:08.157 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:42:08.157 7f4c44296400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:42:08.158 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:08.158 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:42:08.159 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:08.159 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:08.159 7f4c44296400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 21 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:42:08.160 7f4c44296400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:12.158 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:12.160 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:12.160 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:12.160 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:12.160 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:42:12.160 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:12.161 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:12.161 7f4c438c77c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:42:12.161 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:12.161 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:12.162 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:42.161 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:42:42.161 7f4c43877400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:42.162 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:42.163 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 22 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:42:42.163 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:46.166 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:46.168 7f4c43877400 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:46.168 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:46.168 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:46.169 7f4c43877400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:42:46.169 7f4c43877400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:46.169 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:46.169 7f4c43877400 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:42:46.169 7f4c43877400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:46.169 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:46.170 7f4c43877400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:50.564 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:42:50.853 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:00.299 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6935408, partition_space_available_1k=74155076, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319240, cpu_share=0.09, mem_res=1306712, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15788, cpu_share=0.0, mem_res=13880, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56124, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5321980, free=4641684, buffers=211452, cached=6306524, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=3.38, fifteen_min_avg=2.36, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:43:00.300 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:16.171 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:43:16.171 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:16.172 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:16.173 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 23 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:43:16.173 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.175 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:20.176 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:20.177 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:20.177 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:20.177 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:43:20.177 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:20.177 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:20.177 7f4c438c77c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:43:20.178 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:20.178 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:20.178 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:50.183 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:43:50.183 7f4c439b0720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:43:50.184 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:43:50.184 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:43:50.184 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:43:50.184 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.185 7f4c439b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:43:50.185 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.185 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.186 7f4c439b0720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 24 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:43:50.186 7f4c439b0720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.573 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:43:50.828 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.828 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. database_usage = [ << disk_space_used_1k = 6942636 disk_space_available_1k = 74147848 analytics_db_size_1k = 772 >>, ] >> 09/14/2026 02:43:50.828 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:54.185 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:54.187 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:54.187 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:54.187 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:54.187 7f4c438777c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:43:54.187 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:54.187 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:54.188 7f4c438777c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:43:54.188 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:54.188 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:54.188 7f4c438777c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:58.884 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6942840, partition_space_available_1k=74147644, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1307160, cpu_share=0.09, mem_res=1294652, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16020, cpu_share=0.0, mem_res=14100, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56172, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5244856, free=4708436, buffers=212168, cached=6315188, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.5, five_min_avg=3.7, fifteen_min_avg=2.54, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:43:58.884 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:24.196 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:44:24.196 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:44:24.196 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 25 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:44:24.197 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:28.199 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:44:28.200 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:44:28.200 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:44:28.201 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:28.201 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:44:28.201 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:28.201 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:44:28.201 7f4c438c77c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:44:28.201 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:44:28.202 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:28.202 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:50.579 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:44:50.837 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:58.201 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:44:58.202 7f4c439b0a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:44:58.202 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:44:58.203 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:44:58.203 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:44:58.203 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:58.203 7f4c439b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:44:58.203 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:58.203 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:58.204 7f4c439b0a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 26 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:44:58.204 7f4c439b0a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:01.047 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6950504, partition_space_available_1k=74139980, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1299796, cpu_share=0.08, mem_res=1287240, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16588, cpu_share=0.0, mem_res=14664, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56164, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5351548, free=4594220, buffers=212920, cached=6321972, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.21, five_min_avg=3.69, fifteen_min_avg=2.61, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:45:01.047 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:01.048 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6950504 partition_space_available_1k = 74139980 percentage_partition_space_used = 9 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1299796 cpu_share = 0.08 mem_res = 1287240 >>contrail-query-engine : << mem_virt = 16588 cpu_share = 0.0 mem_res = 14664 >>contrail-database-nodemgr : << mem_virt = 56164 cpu_share = 0.01 mem_res = 46380 >> } system_mem_usage = << total = 16175632 used = 5351548 free = 4594220 buffers = 212920 cached = 6321972 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.21 five_min_avg = 3.69 fifteen_min_avg = 2.61 cpu_share = 14.25 node_type = database-node >> >> 09/14/2026 02:45:01.048 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.204 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:02.205 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:02.205 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:02.205 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:02.206 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:45:02.206 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:02.206 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:02.206 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:45:02.206 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:02.206 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:02.207 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:32.210 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:45:32.210 7f4c43877720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:45:32.211 7f4c43877720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:45:32.211 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:45:32.211 7f4c43877720 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:45:32.211 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:32.211 7f4c43877720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:45:32.211 7f4c43877720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:32.212 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:32.212 7f4c43877720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 27 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:45:32.212 7f4c43877720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:36.215 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:36.217 7f4c43877720 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:45:36.218 7f4c43877720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:36.218 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:36.218 7f4c43877720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:50.637 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:45:51.184 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:59.471 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6958012, partition_space_available_1k=74132472, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303380, cpu_share=0.09, mem_res=1290704, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16920, cpu_share=0.0, mem_res=14984, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56180, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5143540, free=4791552, buffers=213600, cached=6331956, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.05, five_min_avg=3.43, fifteen_min_avg=2.58, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:45:59.472 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:06.223 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:46:06.223 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:46:06.224 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:46:06.224 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:46:06.224 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:46:06.224 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:06.224 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:46:06.224 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:06.225 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:06.225 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 28 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:46:06.225 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:10.225 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:10.227 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:10.227 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:10.227 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:10.227 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:46:10.227 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:10.227 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:10.228 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:46:10.228 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:10.228 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:10.228 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:40.235 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:46:40.235 7f4c439b0c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 29 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:46:40.236 7f4c439b0c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:44.238 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:44.240 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:44.240 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:44.240 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:44.240 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:46:44.240 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:44.240 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:44.241 7f4c438c77c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:46:44.241 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:44.241 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:44.241 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:50.674 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:46:51.018 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:59.502 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6965812, partition_space_available_1k=74124672, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321372, cpu_share=0.09, mem_res=1308616, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=17416, cpu_share=0.0, mem_res=15468, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56200, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4764924, free=5165132, buffers=214312, cached=6336156, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.49, fifteen_min_avg=2.66, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:46:59.504 7f4c44296400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:14.241 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:47:14.241 7f4c438772c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 30 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:47:14.242 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:18.246 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:18.247 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:18.248 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:18.248 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:18.248 7f4c438772c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:47:18.248 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:18.248 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:18.248 7f4c438772c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:47:18.249 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:18.249 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:18.249 7f4c438772c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:48.253 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:47:48.253 7f4c439b0900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:47:48.254 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:47:48.254 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:47:48.254 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:47:48.254 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:48.255 7f4c439b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:47:48.255 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:48.255 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:48.255 7f4c439b0900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 31 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:47:48.255 7f4c439b0900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:50.582 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:47:50.811 7f4c439b0680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:50.811 7f4c439b0680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. database_usage = [ << disk_space_used_1k = 6972384 disk_space_available_1k = 74118100 analytics_db_size_1k = 772 >>, ] >> 09/14/2026 02:47:50.811 7f4c439b0680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:52.255 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:52.257 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:52.257 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:52.257 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:52.258 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:47:52.258 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:52.258 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:52.258 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:47:52.259 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:52.259 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:52.259 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:59.501 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6972940, partition_space_available_1k=74117544, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1312176, cpu_share=0.09, mem_res=1299468, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=17792, cpu_share=0.0, mem_res=15844, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56216, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5157508, free=4762380, buffers=214856, cached=6345800, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.92, five_min_avg=3.29, fifteen_min_avg=2.64, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:47:59.502 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:22.265 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:48:22.266 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:48:22.266 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:48:22.266 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:48:22.266 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:48:22.266 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:22.266 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:48:22.267 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:22.267 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:22.267 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 32 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:48:22.267 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.270 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:48:26.271 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:48:26.271 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:48:26.271 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:26.272 7f4c439b0040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:50.768 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:48:51.124 7f4c438774a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:56.272 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:48:56.273 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:48:56.273 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:48:56.274 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:48:56.274 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:48:56.274 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:56.274 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:48:56.274 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:56.274 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:56.275 7f4c439b0180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 33 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:48:56.275 7f4c439b0180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:59.522 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6980592, partition_space_available_1k=74109892, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1315212, cpu_share=0.15, mem_res=1302444, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=18168, cpu_share=0.0, mem_res=16212, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56228, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5479660, free=4429072, buffers=215360, cached=6356448, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.8, five_min_avg=3.18, fifteen_min_avg=2.64, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:48:59.522 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:59.523 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6980592 partition_space_available_1k = 74109892 percentage_partition_space_used = 9 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1315212 cpu_share = 0.15 mem_res = 1302444 >>contrail-query-engine : << mem_virt = 18168 cpu_share = 0.0 mem_res = 16212 >>contrail-database-nodemgr : << mem_virt = 56228 cpu_share = 0.01 mem_res = 46380 >> } system_mem_usage = << total = 16175632 used = 5479660 free = 4429072 buffers = 215360 cached = 6356448 node_type = database-node >> system_cpu_usage = << one_min_avg = 2.8 five_min_avg = 3.18 fifteen_min_avg = 2.64 cpu_share = 14.0 node_type = database-node >> >> 09/14/2026 02:48:59.523 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:00.275 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:49:00.277 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:49:00.277 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:49:00.278 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:49:00.278 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:49:00.278 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:49:00.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:49:00.279 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:49:00.280 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:49:00.280 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:00.280 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:30.278 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:49:30.278 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:30.279 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:30.280 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 34 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:49:30.280 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:34.283 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:49:34.284 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:49:34.284 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:49:34.285 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:34.286 7f4c44227d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:50.641 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:49:51.139 7f4c43877e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:59.475 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6988420, partition_space_available_1k=74102064, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1309424, cpu_share=0.1, mem_res=1296700, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=18384, cpu_share=0.0, mem_res=16412, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56240, cpu_share=0.01, mem_res=46380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5169792, free=4731684, buffers=215896, cached=6363140, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.54, five_min_avg=3.27, fifteen_min_avg=2.7, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:49:59.476 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:50:04.288 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:50:04.289 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:50:04.289 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:50:04.290 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:50:04.290 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:50:04.290 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:04.290 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:50:04.290 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:04.291 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:04.291 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 35 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:50:04.291 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:08.294 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:50:08.295 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:50:08.295 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:50:08.295 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:50:08.295 7f4c438c77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:50:08.296 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:50:08.296 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:50:08.296 7f4c438c77c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:50:08.296 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:50:08.296 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:50:08.296 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:50:38.303 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:50:38.303 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:38.304 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:38.305 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 36 pid = 1 http_port = 8103 start_time = 1789352989942962 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.189:8086, 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:50:38.305 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:50:40.966 7f4c439b00e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/14/2026 02:50:42.306 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:50:42.307 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:50:42.307 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:50:42.307 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:50:42.307 7f4c439b00e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-3. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:50:42.308 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:50:42.308 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:50:42.308 7f4c439b00e0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:50:42.308 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:50:42.308 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:50:42.308 7f4c439b00e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:50:50.791 7f4c46a3c840 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:50:51.315 7f4c438c77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]