09/14/2026 02:29:49.963 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/14/2026 02:29:49.963 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/14/2026 02:29:49.963 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/14/2026 02:29:49.974 7fc0704e0900 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 09/14/2026 02:29:49.977 7fc0704e0900 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 09/14/2026 02:29:49.978 7fc0704e0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/14/2026 02:29:49.980 7fc0704e0cc0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/14/2026 02:29:49.981 7fc0704e0cc0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/14/2026 02:29:49.981 7fc0704e0cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/14/2026 02:29:49.981 7fc0704e0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:49.981 7fc0704e0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:49.982 7fc0704e0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:49.982 7fc0704e0cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:49.982 7fc0704e0cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/14/2026 02:29:49.982 7fc0704e0cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.313 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 110883913740846010999358800794936258586852060937659800713068564123823298593182, 'start': '1789352985000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 16048624542383323458402647754557308970206754556873412099873999313206908351368, 'start': '1789352986000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 56740559236615443897097149086192291597471352771502451825732827237283091836405, 'start': '1789352986000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/14/2026 02:29:50.387 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.388 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> >> 09/14/2026 02:29:50.388 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.388 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-1.', _node_type='Database', _instance_id='0', _timestamp=1789352990388823, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:50.389 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-1.', _node_type='Database', _instance_id='0', _timestamp=1789352990389181, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352985000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:50.391 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.392 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:50.392 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.392 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:50.392 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352985000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]} >> 09/14/2026 02:29:50.392 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:51.372 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:29:52.266 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/14/2026 02:29:52.269 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:52.269 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. database_usage = [ << disk_space_used_1k = 11712772 disk_space_available_1k = 69377712 analytics_db_size_1k = 4 >>, ] >> 09/14/2026 02:29:52.270 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.296 7fc07380e5c0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/14/2026 02:29:53.296 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-1.', _node_type='Database', _instance_id='0', _timestamp=1789352993296571, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:29:53.298 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.299 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/14/2026 02:29:53.299 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.981 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:53.983 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:29:53.983 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:53.983 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:53.984 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:29:53.984 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:53.984 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:29:53.984 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:29:53.984 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.984 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:29:53.985 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.985 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:53.985 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:29:53.985 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:56.812 7fc07380e5c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 09/14/2026 02:29:57.984 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:29:57.986 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:29:57.986 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:29:57.986 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:57.986 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:29:57.986 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:29:57.986 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:29:57.987 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:29:57.987 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.988 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:29:57.988 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.988 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:29:57.988 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:29:57.988 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.988 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:01.990 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:01.990 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:01.991 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:01.991 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:01.991 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:01.992 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:01.992 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:01.993 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.993 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:01.993 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.993 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:01.994 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:01.994 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:02.120 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-1.', _node_type='Database', _instance_id='0', _timestamp=1789353002120860, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352985000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1789352986000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:30:02.121 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11815928, partition_space_available_1k=69274556, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1284832, cpu_share=0.0, mem_res=1276760, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=8060, cpu_share=0.0, mem_res=6328, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53528, cpu_share=0.0, mem_res=44220, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7604624, free=230032, buffers=463476, cached=8290984, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.26, five_min_avg=3.6, fifteen_min_avg=1.87, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:30:02.121 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:02.121 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352985000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1789352986000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-09-14 00:57:02.446689", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15999-1.novalocal", "build-id": "v0+nightly", "build-number": "202609140010"}]} >> 09/14/2026 02:30:02.121 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:02.121 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:02.121 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 11815928 partition_space_available_1k = 69274556 percentage_partition_space_used = 15 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1284832 cpu_share = 0.0 mem_res = 1276760 >>contrail-query-engine : << mem_virt = 8060 cpu_share = 0.0 mem_res = 6328 >>contrail-database-nodemgr : << mem_virt = 53528 cpu_share = 0.0 mem_res = 44220 >> } system_mem_usage = << total = 16175636 used = 7604624 free = 230032 buffers = 463476 cached = 8290984 node_type = database-node >> system_cpu_usage = << one_min_avg = 7.26 five_min_avg = 3.6 fifteen_min_avg = 1.87 cpu_share = 0 node_type = database-node >> >> 09/14/2026 02:30:02.122 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:05.996 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:06.013 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:06.014 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:06.014 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:06.014 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:06.014 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:06.014 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:06.015 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:06.015 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:06.015 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:06.015 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:06.016 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:06.016 7fc0704e0d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:06.023 7fc0704e0d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:10.014 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:10.016 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:10.016 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:10.016 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:10.016 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:10.016 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:10.017 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:10.017 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:10.017 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:10.017 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:10.017 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:10.018 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:10.018 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:10.018 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:14.018 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:14.019 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/14/2026 02:30:14.020 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:14.020 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:14.020 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:14.020 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:14.020 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/14/2026 02:30:14.021 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/14/2026 02:30:14.021 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:14.021 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/14/2026 02:30:14.022 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:14.022 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:14.023 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/14/2026 02:30:14.023 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:18.026 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:18.038 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:18.038 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:18.038 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:18.039 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:18.039 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:18.039 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:18.039 7fc06f9572c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:30:18.039 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:18.040 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:18.040 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:48.038 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:30:48.038 7fc07063f400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:30:48.039 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:30:48.039 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:30:48.039 7fc07063f400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:30:48.039 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:48.039 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:30:48.039 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:48.040 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:48.040 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 09/14/2026 02:30:48.040 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:30:50.852 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:30:52.040 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:30:52.042 7fc07063f400 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:30:52.043 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:30:52.043 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:52.043 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:30:52.044 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:30:52.044 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:30:52.044 7fc07063f400 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:30:52.044 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:30:52.044 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:52.045 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:52.396 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:53.535 7fc07380e5c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-1.', _node_type='Database', _instance_id='0', _timestamp=1789353053535747, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.189:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:30:53.537 7fc06f957ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:30:59.988 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-6277-1.', _node_type='Database', _instance_id='0', _timestamp=1789353059988192, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 09/14/2026 02:30:59.988 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 09/14/2026 02:30:59.994 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:05.723 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12676220, partition_space_available_1k=68414264, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1297032, cpu_share=0.2, mem_res=1284492, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10816, cpu_share=0.0, mem_res=8992, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52908, cpu_share=0.02, mem_res=44408, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8266416, free=515416, buffers=479080, cached=7340108, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.66, five_min_avg=4.66, fifteen_min_avg=2.35, cpu_share=21.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:31:05.724 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:22.055 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:31:22.056 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:31:22.056 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:31:22.056 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:31:22.056 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:31:22.056 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:22.057 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:31:22.057 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:22.057 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:22.057 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 2 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:31:22.057 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:26.058 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:31:26.060 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:31:26.060 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:31:26.060 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:26.060 7fc07063fa40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:31:26.060 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:31:26.060 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:31:26.061 7fc07063fa40 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:31:26.061 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:31:26.061 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:26.061 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:50.759 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:31:51.154 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:31:56.060 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:31:56.061 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:31:56.062 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:00.062 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:00.064 7fc06fc58c20 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:32:00.065 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:00.065 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:00.065 7fc06fc58c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:00.869 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12681068, partition_space_available_1k=68409416, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1292736, cpu_share=0.14, mem_res=1280304, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11300, cpu_share=0.0, mem_res=9464, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53032, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8216460, free=654392, buffers=479224, cached=7251168, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.64, five_min_avg=4.83, fifteen_min_avg=2.54, cpu_share=16.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:32:00.869 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:30.067 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.068 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:30.069 7fc07063fa40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:32:30.069 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:32:34.070 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:32:34.072 7fc07063fa40 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:32:34.073 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:32:34.073 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:34.073 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:32:50.768 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:32:51.271 7fc07063fa40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:02.015 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12688192, partition_space_available_1k=68402292, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1320688, cpu_share=0.11, mem_res=1308244, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11544, cpu_share=0.0, mem_res=9704, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53024, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8058844, free=804028, buffers=479484, cached=7258884, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.88, five_min_avg=4.61, fifteen_min_avg=2.61, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:33:02.015 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:04.073 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:33:04.074 7fc06f957220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:04.075 7fc06f957220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:33:04.076 7fc06f957220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:08.078 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:08.079 7fc06f957220 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:08.080 7fc06f957220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:08.080 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:08.080 7fc06f957220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:33:08.080 7fc06f957220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:08.081 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:08.081 7fc06f957220 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:33:08.081 7fc06f957220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:08.081 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:08.082 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:38.093 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:33:38.093 7fc06f957ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:33:38.094 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:33:38.094 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:33:38.094 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:33:38.094 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:38.094 7fc06f957ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:33:38.095 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:38.095 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:38.095 7fc06f957ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:33:38.095 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:33:42.096 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:33:42.098 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:33:42.098 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:33:42.098 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:42.098 7fc06f957ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:33:42.098 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:33:42.099 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:33:42.099 7fc06f957ea0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:33:42.099 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:33:42.099 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:42.099 7fc06f957ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:33:50.666 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:33:51.038 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:01.061 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12704636, partition_space_available_1k=68385848, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1295800, cpu_share=0.1, mem_res=1283344, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12156, cpu_share=0.0, mem_res=10304, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53036, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8459924, free=387208, buffers=479768, cached=7274348, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.84, five_min_avg=4.61, fifteen_min_avg=2.74, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:34:01.061 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:12.098 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:34:12.098 7fc06f957040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:12.099 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:12.100 7fc06f957040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 7 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:34:12.100 7fc06f957040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:16.103 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:16.105 7fc06f957040 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:16.105 7fc06f957040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:16.105 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:16.105 7fc06f957040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:34:16.105 7fc06f957040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:16.105 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:16.106 7fc06f957040 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:34:16.106 7fc06f957040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:16.106 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:16.106 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:46.108 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:34:46.109 7fc06fc580e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:34:46.109 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:34:46.109 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:34:46.109 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:34:46.110 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:46.110 7fc06fc580e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:34:46.110 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:46.110 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:46.110 7fc06fc580e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 8 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:34:46.110 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:34:50.110 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:34:50.111 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:34:50.111 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:34:50.111 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:50.112 7fc06fc580e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:50.679 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:34:51.050 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:34:59.821 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12711964, partition_space_available_1k=68378520, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342660, cpu_share=0.11, mem_res=1330180, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12372, cpu_share=0.0, mem_res=10508, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53080, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7810284, free=1030576, buffers=479892, cached=7279420, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.2, five_min_avg=4.53, fifteen_min_avg=2.83, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:34:59.821 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:20.118 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:35:20.118 7fc07063f400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:20.119 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:20.120 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 9 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:35:20.120 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:24.123 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:24.124 7fc07063f400 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:24.125 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:24.125 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:24.125 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:35:24.125 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:24.125 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:24.125 7fc07063f400 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:35:24.126 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:24.126 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:24.126 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:50.785 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:35:51.324 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:54.126 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:35:54.126 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:35:54.127 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:35:54.127 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:35:54.127 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:35:54.127 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:54.128 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:35:54.128 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:54.128 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:54.128 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 10 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:35:54.128 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:35:58.128 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:35:58.130 7fc06f957900 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:35:58.131 7fc06f957900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:35:58.131 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:58.131 7fc06f957900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:35:58.131 7fc06f957900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:35:58.132 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:35:58.132 7fc06f957900 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:35:58.133 7fc06f957900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:35:58.133 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:35:58.133 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:02.046 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12720604, partition_space_available_1k=68369880, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1308112, cpu_share=0.1, mem_res=1295332, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12916, cpu_share=0.0, mem_res=11044, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53112, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8452344, free=407096, buffers=479452, cached=7261500, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.43, five_min_avg=4.68, fifteen_min_avg=2.98, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:36:02.046 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:28.130 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:36:28.130 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 11 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:36:28.131 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:36:32.135 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:36:32.136 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:36:32.137 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:36:32.137 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:32.137 7fc0704e0220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:36:32.137 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:36:32.137 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:36:32.137 7fc0704e0220 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:36:32.138 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:36:32.138 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:32.138 7fc0704e0220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:50.681 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:36:51.134 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:36:59.832 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12728088, partition_space_available_1k=68362396, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1308084, cpu_share=0.1, mem_res=1295304, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13288, cpu_share=0.0, mem_res=11408, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53104, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8289184, free=561616, buffers=479604, cached=7269020, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.36, five_min_avg=4.46, fifteen_min_avg=3.01, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:36:59.833 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:02.138 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:37:02.139 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:37:02.139 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:37:02.139 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:37:02.139 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:37:02.139 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:02.139 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:37:02.140 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:02.140 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:02.140 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 12 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:37:02.140 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:06.143 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:06.144 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:06.145 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:06.145 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:06.145 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:37:06.145 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:06.146 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:06.146 7fc06fb817c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:37:06.146 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:06.146 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:06.147 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:36.150 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:37:36.150 7fc06f957900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:36.151 7fc06f957900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 13 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:37:36.152 7fc06f957900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:37:40.154 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:37:40.155 7fc06f957900 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:37:40.156 7fc06f957900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:37:40.156 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:40.156 7fc06f957900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:37:40.156 7fc06f957900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:37:40.156 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:37:40.156 7fc06f957900 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:37:40.157 7fc06f957900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:37:40.157 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:40.157 7fc06f957900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:37:51.121 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:37:51.819 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:02.524 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12735956, partition_space_available_1k=68354528, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1328556, cpu_share=0.1, mem_res=1315816, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13756, cpu_share=0.0, mem_res=11868, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53120, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8193868, free=674488, buffers=479000, cached=7252144, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.96, five_min_avg=4.57, fifteen_min_avg=3.13, cpu_share=16.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:38:02.524 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:10.162 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:38:10.163 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:38:10.163 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:38:10.163 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:38:10.163 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:38:10.164 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:10.164 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:38:10.164 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:10.164 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:10.164 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 14 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:38:10.164 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:14.165 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:14.167 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:14.167 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:14.167 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:14.167 7fc06fc58360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:38:14.167 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:14.167 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:14.168 7fc06fc58360 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:38:14.168 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:14.168 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:14.168 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:44.168 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:38:44.168 7fc06f957220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:38:44.168 7fc06f957220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:38:44.168 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 15 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:38:44.169 7fc06f957220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:38:48.170 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:38:48.174 7fc06f957220 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:38:48.174 7fc06f957220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:38:48.174 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:48.174 7fc06f957220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:38:48.174 7fc06f957220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:38:48.175 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:38:48.175 7fc06f957220 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:38:48.175 7fc06f957220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:38:48.175 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:48.175 7fc06f957220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:38:50.802 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:38:51.242 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:01.792 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12743664, partition_space_available_1k=68346820, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337420, cpu_share=0.1, mem_res=1324644, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14052, cpu_share=0.0, mem_res=12156, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53140, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8317168, free=537468, buffers=479420, cached=7265440, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.92, five_min_avg=4.61, fifteen_min_avg=3.23, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:39:01.793 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:18.175 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:39:18.175 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:39:18.176 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:39:18.176 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:39:18.176 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:39:18.176 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.176 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:39:18.176 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.177 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:18.177 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 16 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:39:18.177 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:22.178 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:22.179 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:22.180 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:22.180 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:22.180 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:39:22.180 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:22.181 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:22.181 7fc06fc58720 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:39:22.181 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:22.182 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:22.182 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:50.939 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:39:51.800 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:52.179 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:39:52.179 7fc07063f400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 17 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:39:52.180 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:39:56.181 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:39:56.183 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:39:56.183 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:39:56.183 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:56.183 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:39:56.184 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:39:56.184 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:39:56.184 7fc06f9572c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:39:56.184 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:39:56.184 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:39:56.184 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:00.484 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12752000, partition_space_available_1k=68338484, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1326568, cpu_share=0.09, mem_res=1313756, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14604, cpu_share=0.0, mem_res=12700, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53168, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8117392, free=770276, buffers=479104, cached=7232900, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.26, five_min_avg=4.46, fifteen_min_avg=3.26, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:40:00.484 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:26.194 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:40:26.195 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:40:26.195 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:40:26.195 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:40:26.196 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:40:26.196 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:26.196 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:40:26.196 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:26.197 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:26.197 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 18 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:40:26.197 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:40:30.196 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:40:30.197 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:40:30.198 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:40:30.198 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:30.198 7fc06fb81540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:40:30.199 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:40:30.199 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:40:30.199 7fc06fb81540 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:40:30.200 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:40:30.200 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:30.201 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:40:50.750 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:40:51.289 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:00.199 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:41:00.200 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:41:00.200 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:41:00.200 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:41:00.200 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:41:00.200 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.201 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:41:00.201 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.201 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.201 7fc06fc58720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 19 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:41:00.201 7fc06fc58720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.696 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12759576, partition_space_available_1k=68330908, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327852, cpu_share=0.1, mem_res=1314908, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14864, cpu_share=0.0, mem_res=12948, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53160, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8249860, free=620504, buffers=479520, cached=7250784, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.49, five_min_avg=5.15, fifteen_min_avg=3.58, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:41:00.696 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:00.696 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12759576 partition_space_available_1k = 68330908 percentage_partition_space_used = 16 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1327852 cpu_share = 0.1 mem_res = 1314908 >>contrail-query-engine : << mem_virt = 14864 cpu_share = 0.0 mem_res = 12948 >>contrail-database-nodemgr : << mem_virt = 53160 cpu_share = 0.02 mem_res = 44516 >> } system_mem_usage = << total = 16175636 used = 8249860 free = 620504 buffers = 479520 cached = 7250784 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.49 five_min_avg = 5.15 fifteen_min_avg = 3.58 cpu_share = 15.25 node_type = database-node >> >> 09/14/2026 02:41:00.697 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:04.202 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:04.204 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:04.204 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:04.204 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:04.204 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:41:04.204 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:04.205 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:04.205 7fc06fb817c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:41:04.205 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:04.205 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:04.205 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:34.208 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:41:34.208 7fc06f957040 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:41:34.208 7fc06f957040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 20 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:41:34.209 7fc06f957040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:41:38.212 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:41:38.213 7fc06f957040 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:41:38.214 7fc06f957040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:41:38.214 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:38.214 7fc06f957040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:41:38.214 7fc06f957040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:41:38.214 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:41:38.214 7fc06f957040 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:41:38.215 7fc06f957040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:41:38.215 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:38.215 7fc06f957040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:41:50.908 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:41:51.266 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:01.267 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12766884, partition_space_available_1k=68323600, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1322132, cpu_share=0.09, mem_res=1308992, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15392, cpu_share=0.0, mem_res=13464, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53168, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8315292, free=573556, buffers=479324, cached=7231560, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.51, five_min_avg=5.18, fifteen_min_avg=3.7, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:42:01.267 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:08.218 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:42:08.218 7fc06f957860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:42:08.219 7fc06f957860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:42:08.219 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:42:08.219 7fc06f957860 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:42:08.219 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:08.219 7fc06f957860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:42:08.219 7fc06f957860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:08.220 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:08.220 7fc06f957860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 21 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:42:08.220 7fc06f957860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:12.222 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:12.224 7fc06f957860 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:12.224 7fc06f957860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:12.224 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:12.225 7fc06f957860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:42:12.225 7fc06f957860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:12.225 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:12.225 7fc06f957860 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:42:12.227 7fc06f957860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:12.227 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:12.227 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:42.230 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:42:42.231 7fc06f957860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:42:42.231 7fc06f957860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:42:42.231 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 22 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:42:42.232 7fc06f957860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:42:46.236 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:42:46.237 7fc06f957860 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:42:46.237 7fc06f957860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:42:46.237 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:46.237 7fc06f957860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:42:46.238 7fc06f957860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:42:46.238 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:42:46.238 7fc06f957860 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:42:46.238 7fc06f957860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:42:46.238 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:46.239 7fc06f957860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:42:50.755 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:42:51.129 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:02.057 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12775124, partition_space_available_1k=68315360, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1317800, cpu_share=0.1, mem_res=1304788, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15688, cpu_share=0.0, mem_res=13752, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53176, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8378364, free=505532, buffers=479732, cached=7241440, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.06, five_min_avg=5.65, fifteen_min_avg=3.95, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:43:02.058 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:16.251 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:43:16.251 7fc06fb81540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:43:16.252 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:43:16.252 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:43:16.252 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:43:16.252 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:16.253 7fc06fb81540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:43:16.253 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:16.253 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:16.253 7fc06fb81540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 23 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:43:16.253 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:20.256 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:20.257 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:20.258 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:20.258 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:20.258 7fc06fc584a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:43:20.258 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:20.258 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:20.258 7fc06fc584a0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:43:20.259 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:20.259 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:20.259 7fc06fc584a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:50.259 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:43:50.259 7fc06f957f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:43:50.259 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 24 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:43:50.260 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:51.033 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:43:51.483 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:51.483 7fc06f957f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. database_usage = [ << disk_space_used_1k = 12781852 disk_space_available_1k = 68308632 analytics_db_size_1k = 772 >>, ] >> 09/14/2026 02:43:51.484 7fc06f957f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:43:54.261 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:43:54.262 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:43:54.262 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:43:54.263 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:54.263 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:43:54.263 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:43:54.263 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:43:54.263 7fc06f957d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:43:54.264 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:43:54.264 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:43:54.264 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:00.119 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12782224, partition_space_available_1k=68308260, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1307500, cpu_share=0.09, mem_res=1294560, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16144, cpu_share=0.0, mem_res=14188, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53196, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8248244, free=627728, buffers=479848, cached=7243096, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.17, five_min_avg=5.36, fifteen_min_avg=3.96, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:44:00.120 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:24.267 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:44:24.267 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:24.268 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 25 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:44:24.269 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:28.270 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:44:28.272 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:44:28.272 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:44:28.272 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:28.273 7fc06f957d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:44:28.273 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:44:28.273 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:44:28.273 7fc06f957d60 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:44:28.274 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:44:28.274 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:28.274 7fc06f957d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:50.989 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:44:51.932 7fc06f9577c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:44:58.273 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:44:58.274 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:44:58.274 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:44:58.275 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:44:58.275 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:44:58.275 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:58.276 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:44:58.276 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:58.276 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:44:58.277 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 26 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:44:58.277 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:02.275 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:02.277 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:02.279 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:02.279 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:02.279 7fc06f9572c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:45:02.279 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:02.279 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:02.280 7fc06f9572c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:45:02.280 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:02.280 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:02.281 7fc06f9572c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:04.072 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12790936, partition_space_available_1k=68299548, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1307952, cpu_share=0.09, mem_res=1294928, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16516, cpu_share=0.0, mem_res=14564, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53240, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8228992, free=637876, buffers=479932, cached=7252124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.24, five_min_avg=5.25, fifteen_min_avg=4.0, cpu_share=16.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:45:04.073 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:32.290 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:45:32.291 7fc06fc58900 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:45:32.291 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 27 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:45:32.292 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:45:36.294 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:45:36.295 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:45:36.296 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:45:36.296 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:36.296 7fc06fc58900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:45:36.296 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:45:36.296 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:45:36.297 7fc06fc58900 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:45:36.297 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:45:36.297 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:36.297 7fc06fc58900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:45:50.704 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:45:51.034 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:00.011 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12798064, partition_space_available_1k=68292420, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305760, cpu_share=0.09, mem_res=1292600, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16952, cpu_share=0.0, mem_res=14988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53224, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8480756, free=390940, buffers=479996, cached=7247288, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.61, five_min_avg=5.31, fifteen_min_avg=4.1, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:46:00.012 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:06.297 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:46:06.297 7fc06f957cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:06.298 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:06.299 7fc06f957cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 28 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:46:06.299 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:10.302 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:10.305 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:10.305 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:10.306 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:10.306 7fc06f957cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:46:10.306 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:10.306 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:10.306 7fc06f957cc0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:46:10.307 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:10.307 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:10.307 7fc06f957cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:40.309 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:46:40.309 7fc06fc58360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:40.310 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:40.311 7fc06fc58360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 29 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:46:40.311 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:46:44.314 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:46:44.316 7fc06fc58360 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:46:44.317 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:46:44.317 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:44.317 7fc06fc58360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:46:50.790 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:46:51.231 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:00.823 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12805908, partition_space_available_1k=68284576, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1300348, cpu_share=0.09, mem_res=1287128, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=17308, cpu_share=0.0, mem_res=15340, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53216, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7984928, free=577600, buffers=479936, cached=7556824, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.71, five_min_avg=5.08, fifteen_min_avg=4.09, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:47:00.825 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:14.318 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:47:14.319 7fc06f957c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:47:14.319 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:47:14.320 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:47:14.320 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:47:14.320 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:14.320 7fc06f957c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:47:14.320 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:14.321 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:14.321 7fc06f957c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 30 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:47:14.321 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:18.320 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:18.321 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:47:18.322 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:18.323 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:18.323 7fc06f957c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:48.330 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:47:48.330 7fc07063f400 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 31 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:47:48.331 7fc07063f400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:50.885 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:47:51.427 7fc06f957400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:51.428 7fc06f957400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. database_usage = [ << disk_space_used_1k = 13490024 disk_space_available_1k = 67600460 analytics_db_size_1k = 772 >>, ] >> 09/14/2026 02:47:51.428 7fc06f957400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:47:52.332 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:47:52.333 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:47:52.333 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:47:52.333 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:52.334 7fc06fb81540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:47:52.334 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:47:52.334 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:47:52.334 7fc06fb81540 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:47:52.335 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:47:52.335 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:47:52.336 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:03.398 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13490904, partition_space_available_1k=67599580, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1331016, cpu_share=0.12, mem_res=1317988, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=17888, cpu_share=0.0, mem_res=15908, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53024, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8515100, free=359596, buffers=480008, cached=7249244, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.0, five_min_avg=5.05, fifteen_min_avg=4.14, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:48:03.399 7fc06fb81540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:22.334 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:48:22.334 7fc06fc58ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:48:22.335 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:48:22.335 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:48:22.335 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:48:22.335 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:22.336 7fc06fc58ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.189:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:48:22.336 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:22.336 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:22.336 7fc06fc58ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 32 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.189:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:48:22.336 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:26.338 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:48:26.340 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:48:26.340 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:48:26.340 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:26.340 7fc06fc58ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:48:26.340 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:48:26.340 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:48:26.341 7fc06fc58ea0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:48:26.341 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:48:26.341 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:26.342 7fc06fc58ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:50.808 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/14/2026 02:48:51.155 7fc07063f400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:48:56.340 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvConnectTimerExpired] in state[ClientInit] 09/14/2026 02:48:56.340 7fc06fc58ae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/14/2026 02:48:56.341 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvConnectTimerExpired] => State[ClientInit] -> State[Idle] 09/14/2026 02:48:56.341 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Idle] 09/14/2026 02:48:56.341 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Ignore event [EvTcpClose] received for old session 09/14/2026 02:48:56.341 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:56.341 7fc06fc58ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvConnectTimerExpired >>, ] description = Collector:Collector[ClientInit to Idle on EvConnectTimerExpired] connection down >>, ] >> 09/14/2026 02:48:56.342 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:56.342 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:56.342 7fc06fc58ae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 33 pid = 1 http_port = 8103 start_time = 1789352989980846 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.189:8086, 10.0.0.38:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 3 >> 09/14/2026 02:48:56.342 7fc06fc58ae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:59.277 7fc06f957ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/14/2026 02:48:59.888 7fc07380e5c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-6277-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13610472, partition_space_available_1k=67480012, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327560, cpu_share=0.09, mem_res=1314500, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=18212, cpu_share=0.0, mem_res=16224, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53012, cpu_share=0.02, mem_res=44516, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8122200, free=617760, buffers=483740, cached=7381224, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=4.61, fifteen_min_avg=4.05, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/14/2026 02:48:59.888 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:48:59.889 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13610472 partition_space_available_1k = 67480012 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1327560 cpu_share = 0.09 mem_res = 1314500 >>contrail-query-engine : << mem_virt = 18212 cpu_share = 0.0 mem_res = 16224 >>contrail-database-nodemgr : << mem_virt = 53012 cpu_share = 0.02 mem_res = 44516 >> } system_mem_usage = << total = 16175636 used = 8122200 free = 617760 buffers = 483740 cached = 7381224 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.49 five_min_avg = 4.61 fifteen_min_avg = 4.05 cpu_share = 15.75 node_type = database-node >> >> 09/14/2026 02:48:59.889 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/14/2026 02:49:00.341 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/14/2026 02:49:00.343 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/14/2026 02:49:00.343 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/14/2026 02:49:00.343 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:49:00.343 7fc06fb817c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-6277-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/14/2026 02:49:00.343 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/14/2026 02:49:00.343 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/14/2026 02:49:00.344 7fc06fb817c0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/14/2026 02:49:00.344 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/14/2026 02:49:00.344 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/14/2026 02:49:00.344 7fc06fb817c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]