12/08/2025 02:05:02.278 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 12/08/2025 02:05:02.278 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 12/08/2025 02:05:02.278 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 12/08/2025 02:05:02.288 7f8969a01360 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 12/08/2025 02:05:02.288 7f8969a01360 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 12/08/2025 02:05:02.289 7f8969a01720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 12/08/2025 02:05:02.289 7f8969a01720 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 12/08/2025 02:05:02.290 7f8969a01720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.398 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 67170245184051509475998645163355710191580118457136910941757713044625971892719, 'start': '1765159500000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 60036568711445691299921827411633782039208328541547943386231960005121950493648, 'start': '1765159500000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 101097333993414007927771559924171655912901383918870459840599542939269715427828, 'start': '1765159500000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 12/08/2025 02:05:02.434 7f8969d0c180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.434 7f8969d0c180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-08 00:51:10.438271", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12945-1.novalocal", "build-id": "v0+nightly", "build-number": "202512080007"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> >> 12/08/2025 02:05:02.435 7f8969d0c180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.435 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4851-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1765159502435377, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 12/08/2025 02:05:02.435 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4851-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1765159502435691, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1765159500000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1765159500000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1765159500000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-08 00:51:10.438271", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12945-1.novalocal", "build-id": "v0+nightly", "build-number": "202512080007"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 12/08/2025 02:05:02.437 7f8969d0c180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.437 7f8969d0c180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/08/2025 02:05:02.438 7f8969d0c180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.438 7f8969d0c180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.438 7f8969d0c180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1765159500000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1765159500000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1765159500000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-08 00:51:10.438271", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12945-1.novalocal", "build-id": "v0+nightly", "build-number": "202512080007"}]} >> 12/08/2025 02:05:02.438 7f8969d0c180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.665 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:05:02.910 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 12/08/2025 02:05:02.911 7f8969ced040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:02.911 7f8969ced040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. database_usage = [ << disk_space_used_1k = 13546500 disk_space_available_1k = 67543984 config_db_size_1k = 108 >>, ] >> 12/08/2025 02:05:02.912 7f8969ced040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:03.279 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 12/08/2025 02:05:03.279 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4851-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1765159503279561, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 12/08/2025 02:05:03.281 7f8969a01e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:03.282 7f8969a01e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 12/08/2025 02:05:03.282 7f8969a01e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:04.298 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:05:05.355 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 12/08/2025 02:05:06.269 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 12/08/2025 02:05:06.289 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:06.291 7f8969ced860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:06.292 7f8969ced860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:06.293 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:06.293 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:06.293 7f8969ced860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:06.294 7f8969ced860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:07.067 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 12/08/2025 02:05:07.910 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 12/08/2025 02:05:08.714 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 12/08/2025 02:05:09.506 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 12/08/2025 02:05:10.293 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:10.295 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:10.295 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:10.295 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:10.295 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:10.295 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:10.295 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:10.296 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.297 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:14.298 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.299 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:14.300 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.300 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.300 7f8969d0c7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:14.300 7f8969d0c7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.538 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4851-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1765159514538558, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1765159500000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1765159500000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1765159500000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-08 00:51:10.438271", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12945-1.novalocal", "build-id": "v0+nightly", "build-number": "202512080007"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 12/08/2025 02:05:14.539 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13760012, partition_space_available_1k=67330472, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1299764, cpu_share=0.0, mem_res=1289152, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51304, cpu_share=0.0, mem_res=49004, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51308, cpu_share=0.0, mem_res=41480, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=5316124, free=13485036, buffers=612792, cached=13796576, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.05, five_min_avg=1.57, fifteen_min_avg=1.06, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:05:14.539 7f8969a01d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.540 7f8969a01d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1765159500000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1765159500000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1765159500000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-08 00:51:10.438271", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12945-1.novalocal", "build-id": "v0+nightly", "build-number": "202512080007"}]} >> 12/08/2025 02:05:14.540 7f8969a01d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.540 7f8969a01d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:14.540 7f8969a01d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13760012 partition_space_available_1k = 67330472 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1299764 cpu_share = 0.0 mem_res = 1289152 >>zookeeper : << mem_virt = 51304 cpu_share = 0.0 mem_res = 49004 >>contrail-config-database-nodemgr : << mem_virt = 51308 cpu_share = 0.0 mem_res = 41480 >> } system_mem_usage = << total = 32665116 used = 5316124 free = 13485036 buffers = 612792 cached = 13796576 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.05 five_min_avg = 1.57 fifteen_min_avg = 1.06 cpu_share = 0 node_type = config-database-node >> >> 12/08/2025 02:05:14.540 7f8969a01d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:18.300 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:18.301 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:18.301 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:18.301 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:18.301 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:18.301 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:18.301 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:18.302 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:22.302 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:22.303 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:22.303 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:22.303 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:22.304 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:26.306 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:26.307 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:26.307 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:26.308 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:26.309 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:26.309 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:30.310 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:30.311 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:30.316 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:30.316 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:30.316 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:30.316 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:30.316 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:30.316 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:30.317 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:30.317 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:30.317 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:30.317 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:30.317 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:30.317 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:34.313 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:34.314 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:34.315 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:34.316 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:34.316 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:34.316 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:34.316 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:38.319 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:38.320 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:38.320 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:38.320 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:38.320 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:38.320 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:38.320 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:38.321 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:42.322 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:42.323 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:42.323 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:42.323 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:42.323 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:42.324 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:42.324 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:42.324 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:42.324 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:42.324 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:42.325 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:42.325 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:42.325 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:42.325 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:46.327 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:46.328 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:46.328 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:46.328 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:46.329 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:46.329 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:46.329 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:46.329 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:46.329 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:46.330 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:46.330 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:46.330 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:46.330 7f8969cedc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:46.330 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:50.330 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:50.331 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:50.331 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:50.332 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:50.333 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:50.333 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:54.334 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:54.335 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:54.335 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:54.335 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:54.335 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:54.336 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:58.338 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:05:58.339 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:05:58.340 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:05:58.340 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:58.340 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:05:58.340 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:05:58.340 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:05:58.340 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:05:58.341 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:58.341 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:05:58.341 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:58.341 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:05:58.341 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:05:58.341 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:02.344 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:02.345 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:02.345 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:02.345 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:02.345 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:02.345 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:02.345 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:02.346 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:02.620 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:06:02.879 7f8969ced400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:02.879 7f8969ced400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. database_usage = [ << disk_space_used_1k = 14683852 disk_space_available_1k = 66406632 config_db_size_1k = 1028 >>, ] >> 12/08/2025 02:06:02.879 7f8969ced400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:03.347 7f896c9f5540 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4851-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1765159563347014, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.130:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 12/08/2025 02:06:03.348 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:03.348 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:03.348 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:04.150 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:06:06.347 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:06.348 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:06.348 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:06.348 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:06.348 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:06.349 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:09.182 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14683972, partition_space_available_1k=66406512, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1320540, cpu_share=0.29, mem_res=1306960, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=73844, cpu_share=0.02, mem_res=70556, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51412, cpu_share=0.01, mem_res=41480, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=6236676, free=11817520, buffers=653484, cached=14500464, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.25, five_min_avg=1.68, fifteen_min_avg=1.12, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:06:09.182 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:09.183 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14683972 partition_space_available_1k = 66406512 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1320540 cpu_share = 0.29 mem_res = 1306960 >>zookeeper : << mem_virt = 73844 cpu_share = 0.02 mem_res = 70556 >>contrail-config-database-nodemgr : << mem_virt = 51412 cpu_share = 0.01 mem_res = 41480 >> } system_mem_usage = << total = 32665116 used = 6236676 free = 11817520 buffers = 653484 cached = 14500464 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.25 five_min_avg = 1.68 fifteen_min_avg = 1.12 cpu_share = 3.12 node_type = config-database-node >> >> 12/08/2025 02:06:09.183 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:10.349 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:10.350 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:10.351 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:10.352 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:10.352 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:10.352 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:10.352 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:10.352 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:14.355 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:14.356 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:14.356 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:14.356 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:14.357 7f8969ced220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:14.358 7f8969ced220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:18.357 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:18.358 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:18.358 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:18.358 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:18.358 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:18.358 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:18.358 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:18.359 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:22.362 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:22.363 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:22.363 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:22.363 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:22.364 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:26.366 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:26.367 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:26.367 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:26.367 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:26.368 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:26.369 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:26.369 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:30.370 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:30.371 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:30.371 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:30.371 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:30.372 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:30.373 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:30.373 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:34.373 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:34.374 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:34.374 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:34.375 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:34.376 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:34.376 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:38.379 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:38.380 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:38.381 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:38.381 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:38.381 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:38.381 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:38.381 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:38.381 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:38.382 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:38.382 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:38.382 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:38.382 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:38.382 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:38.382 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:42.382 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:42.383 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:42.384 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:42.384 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:42.384 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:42.384 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:42.384 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:42.385 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:46.386 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:46.388 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:46.390 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:46.390 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:46.390 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:46.390 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:46.390 7f8969ced680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:46.390 7f8969ced680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:50.389 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:50.391 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:50.391 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:50.391 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:50.391 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:50.391 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:50.391 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:50.392 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:54.394 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:54.395 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:54.395 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:54.396 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:54.396 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:54.396 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:54.396 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:54.396 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:54.396 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:54.397 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:54.397 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:54.397 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:54.397 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:54.397 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:58.399 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:06:58.400 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:06:58.400 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:06:58.400 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:58.400 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:06:58.400 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:06:58.401 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:06:58.401 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:06:58.401 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:58.401 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:06:58.401 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:58.401 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:06:58.402 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:06:58.402 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:02.402 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:02.403 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:02.404 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:02.405 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:02.405 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:02.405 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:02.405 7f8969cedae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:02.405 7f8969cedae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:02.670 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:07:02.915 7f8969ced540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:02.916 7f8969ced540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. database_usage = [ << disk_space_used_1k = 16415540 disk_space_available_1k = 64674944 config_db_size_1k = 1028 >>, ] >> 12/08/2025 02:07:02.916 7f8969ced540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:04.314 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:07:06.406 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:06.407 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:06.407 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:06.407 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:06.407 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:06.408 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:06.408 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:06.408 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:06.408 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:06.408 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:06.408 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:06.409 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:06.409 7f8969d0cc20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:06.409 7f8969d0cc20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:09.347 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16415564, partition_space_available_1k=64674920, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1320824, cpu_share=0.05, mem_res=1307572, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=72700, cpu_share=0.0, mem_res=69544, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51500, cpu_share=0.01, mem_res=41480, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=7876260, free=8286456, buffers=659916, cached=16308256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.02, five_min_avg=1.74, fifteen_min_avg=1.18, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:07:09.347 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:09.347 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16415564 partition_space_available_1k = 64674920 percentage_partition_space_used = 20 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1320824 cpu_share = 0.05 mem_res = 1307572 >>zookeeper : << mem_virt = 72700 cpu_share = 0.0 mem_res = 69544 >>contrail-config-database-nodemgr : << mem_virt = 51500 cpu_share = 0.01 mem_res = 41480 >> } system_mem_usage = << total = 32665116 used = 7876260 free = 8286456 buffers = 659916 cached = 16308256 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.02 five_min_avg = 1.74 fifteen_min_avg = 1.18 cpu_share = 2.62 node_type = config-database-node >> >> 12/08/2025 02:07:09.347 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:10.409 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:10.410 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:10.410 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:10.410 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:10.410 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:10.410 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:10.410 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:10.411 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:14.413 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:14.414 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:14.414 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:14.414 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:14.415 7f8969cedea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:14.416 7f8969cedea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:18.415 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:18.417 7f8969a01ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:18.417 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:18.417 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:18.417 7f8969a01ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:18.417 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:18.417 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:18.418 7f8969a01ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:22.421 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:22.422 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:22.422 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:22.422 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:22.423 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:22.424 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:22.424 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:26.426 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:26.427 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:26.427 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:26.427 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:26.428 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:26.429 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:26.429 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:30.430 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:30.431 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:30.431 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:30.431 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:30.432 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:30.433 7f8969ced720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:30.433 7f8969ced720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:34.433 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:34.434 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 12/08/2025 02:07:34.434 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:34.434 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:34.435 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:34.436 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1765159502289916 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 12/08/2025 02:07:34.436 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 12/08/2025 02:07:38.438 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 12/08/2025 02:07:38.439 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 12/08/2025 02:07:38.439 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 12/08/2025 02:07:38.439 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:38.439 7f8969a01b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4851-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 12/08/2025 02:07:38.439 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 12/08/2025 02:07:38.440 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 12/08/2025 02:07:38.440 7f8969a01b80 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 12/08/2025 02:07:38.440 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 12/08/2025 02:07:38.440 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 12/08/2025 02:07:38.440 7f8969a01b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 12/08/2025 02:07:47.459 7f8969cedf40 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 12/08/2025 02:07:47.460 7f8969cedf40 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 12/08/2025 02:07:47.461 7f896a7424a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 12/08/2025 02:07:47.461 7f896a7424a0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 12/08/2025 02:07:47.462 7f896a7424a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 12/08/2025 02:07:47.462 7f896a7424a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 12/08/2025 02:07:47.462 7f896a7424a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/08/2025 02:07:47.462 7f896a7424a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 12/08/2025 02:07:47.462 7f8969ced7c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 12/08/2025 02:07:47.464 7f8969ced7c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 12/08/2025 02:07:58.017 7f8969cedf40 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438 12/08/2025 02:07:58.018 7f8969cedf40 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 12/08/2025 02:07:58.018 7f8969ced2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 12/08/2025 02:07:58.018 7f8969ced2c0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 12/08/2025 02:07:58.018 7f8969ced2c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 12/08/2025 02:07:58.019 7f8969ced2c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 12/08/2025 02:07:58.019 7f8969ced2c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 12/08/2025 02:07:58.019 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 12/08/2025 02:07:58.019 7f8969cedc20 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 12/08/2025 02:08:02.698 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:08:04.552 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:08:09.591 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17638332, partition_space_available_1k=63452152, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1326668, cpu_share=0.05, mem_res=1313300, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=133392, cpu_share=0.04, mem_res=129656, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51532, cpu_share=0.01, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9028284, free=5779468, buffers=687000, cached=17636924, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.88, five_min_avg=2.26, fifteen_min_avg=1.39, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:09:02.663 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:09:04.559 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:09:09.603 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19484028, partition_space_available_1k=61606456, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333540, cpu_share=0.04, mem_res=1306264, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=141020, cpu_share=0.01, mem_res=137600, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=43768, cpu_share=0.01, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9349428, free=18399124, buffers=703188, cached=4680320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.14, five_min_avg=2.61, fifteen_min_avg=1.57, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:09:46.948 7f8969a01d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 12/08/2025 02:09:48.934 7f8969a01d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 12/08/2025 02:10:02.688 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:10:04.540 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:10:09.612 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19674068, partition_space_available_1k=61416416, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1332400, cpu_share=0.04, mem_res=1305096, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=141492, cpu_share=0.0, mem_res=138072, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46980, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9954888, free=17460352, buffers=712216, cached=5004416, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=2.64, fifteen_min_avg=1.65, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:11:02.697 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:11:04.499 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:11:09.533 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20969260, partition_space_available_1k=60121224, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1365176, cpu_share=0.08, mem_res=1337132, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142744, cpu_share=0.0, mem_res=139304, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46984, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10567856, free=15498532, buffers=747128, cached=6318240, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.81, five_min_avg=2.62, fifteen_min_avg=1.7, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:12:02.711 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:12:04.575 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:12:09.611 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21005800, partition_space_available_1k=60084684, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364224, cpu_share=0.21, mem_res=1335856, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=156704, cpu_share=0.02, mem_res=153196, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46988, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10850276, free=15144820, buffers=747304, cached=6388468, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=2.83, fifteen_min_avg=1.84, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:13:02.811 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:13:04.525 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:13:09.565 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21155060, partition_space_available_1k=59935424, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340404, cpu_share=0.07, mem_res=1311980, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=159000, cpu_share=0.0, mem_res=155332, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46988, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10714968, free=15133704, buffers=747348, cached=6534996, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=3.2, fifteen_min_avg=2.03, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:14:02.739 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:14:04.530 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:14:09.565 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21193456, partition_space_available_1k=59897028, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343532, cpu_share=0.14, mem_res=1314648, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=173732, cpu_share=0.01, mem_res=170036, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46984, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11335808, free=14465524, buffers=747732, cached=6586804, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.98, five_min_avg=3.61, fifteen_min_avg=2.24, cpu_share=4.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:15:02.743 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:15:04.990 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:15:10.048 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21246260, partition_space_available_1k=59844224, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348372, cpu_share=0.1, mem_res=1319236, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=191024, cpu_share=0.03, mem_res=187368, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47000, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11268740, free=14476076, buffers=748032, cached=6638264, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.49, five_min_avg=4.2, fifteen_min_avg=2.53, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:16:02.736 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:16:04.797 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:16:09.833 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21279784, partition_space_available_1k=59810700, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356416, cpu_share=0.08, mem_res=1326596, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=193148, cpu_share=0.01, mem_res=189472, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46996, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10886320, free=14837840, buffers=748092, cached=6658648, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.5, five_min_avg=4.03, fifteen_min_avg=2.57, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:17:02.770 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:17:04.619 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:17:09.665 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21301104, partition_space_available_1k=59789380, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356488, cpu_share=0.07, mem_res=1326624, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183332, cpu_share=0.0, mem_res=179648, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47008, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11146644, free=14545524, buffers=748176, cached=6689728, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.14, five_min_avg=3.96, fifteen_min_avg=2.63, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:18:02.772 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:18:04.506 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:18:09.549 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21327308, partition_space_available_1k=59763176, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376620, cpu_share=0.08, mem_res=1346532, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185816, cpu_share=0.0, mem_res=182108, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47008, cpu_share=0.01, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11183880, free=14476608, buffers=748236, cached=6721036, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.13, five_min_avg=3.71, fifteen_min_avg=2.63, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:19:02.825 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:19:04.639 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:19:09.678 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21451324, partition_space_available_1k=59639160, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376236, cpu_share=0.07, mem_res=1346152, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187248, cpu_share=0.0, mem_res=183532, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47008, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11203844, free=14347892, buffers=748308, cached=6829956, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.54, five_min_avg=3.79, fifteen_min_avg=2.73, cpu_share=4.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:20:02.757 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:20:04.490 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:20:09.534 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21457204, partition_space_available_1k=59633280, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1374052, cpu_share=0.05, mem_res=1343908, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187916, cpu_share=0.0, mem_res=184192, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47020, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11040752, free=14503428, buffers=748332, cached=6837392, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.68, five_min_avg=3.52, fifteen_min_avg=2.7, cpu_share=2.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:21:02.772 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 12/08/2025 02:21:04.419 7f896c9f5540 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 12/08/2025 02:21:09.454 7f896c9f5540 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4851-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21522928, partition_space_available_1k=59567556, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364336, cpu_share=0.09, mem_res=1333484, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=190148, cpu_share=0.0, mem_res=186400, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47020, cpu_share=0.02, mem_res=41476, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10864936, free=14598796, buffers=750704, cached=6914392, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.41, five_min_avg=3.32, fifteen_min_avg=2.69, cpu_share=3.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 12/08/2025 02:21:26.051 7f8969a01ae0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map