11/13/2025 01:58:57.969 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/13/2025 01:58:57.970 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/13/2025 01:58:57.970 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/13/2025 01:58:57.978 7fe6cc9c7360 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 11/13/2025 01:58:57.978 7fe6cc9c7360 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 11/13/2025 01:58:57.979 7fe6cc9c7720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 11/13/2025 01:58:57.979 7fe6cc9c7720 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/13/2025 01:58:57.980 7fe6cc9c7720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.089 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 12016880604321406992180277235544005322737267718466364933297820392842887182078, 'start': '1762999136000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 12553688136323471811648106275586460354297744121750776669453440878244235470802, 'start': '1762999136000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 36113812258890042455721535307245904271219537402614901126940528772726921569774, 'start': '1762999136000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 11/13/2025 01:58:58.123 7fe6cccd3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.123 7fe6cccd3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-11-13 00:53:20.682783", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12697-1.novalocal", "build-id": "v0+nightly", "build-number": "202511130009"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> >> 11/13/2025 01:58:58.123 7fe6cccd3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.124 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4715-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762999138124119, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/13/2025 01:58:58.124 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4715-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762999138124437, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762999136000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762999136000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762999136000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2025-11-13 00:53:20.682783", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12697-1.novalocal", "build-id": "v0+nightly", "build-number": "202511130009"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/13/2025 01:58:58.138 7fe6cccd3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.139 7fe6cccd3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:58:58.139 7fe6cccd3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.139 7fe6cccd3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.139 7fe6cccd3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762999136000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762999136000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762999136000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-11-13 00:53:20.682783", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12697-1.novalocal", "build-id": "v0+nightly", "build-number": "202511130009"}]} >> 11/13/2025 01:58:58.139 7fe6cccd3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.400 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 01:58:58.650 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/13/2025 01:58:58.651 7fe6cc9c7cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:58.651 7fe6cc9c7cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. database_usage = [ << disk_space_used_1k = 13556484 disk_space_available_1k = 67534000 config_db_size_1k = 108 >>, ] >> 11/13/2025 01:58:58.652 7fe6cc9c7cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:59.035 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 11/13/2025 01:58:59.035 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4715-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762999139035366, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/13/2025 01:58:59.036 7fe6cccd3540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:58:59.037 7fe6cccd3540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/13/2025 01:58:59.037 7fe6cccd3540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:00.062 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 01:59:01.229 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 11/13/2025 01:59:01.980 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:01.981 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:01.982 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:01.983 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:01.983 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:02.143 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 11/13/2025 01:59:02.925 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 11/13/2025 01:59:03.751 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 11/13/2025 01:59:04.565 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 11/13/2025 01:59:05.446 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 11/13/2025 01:59:05.982 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:05.984 7fe6cccb4c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:05.984 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:05.984 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:05.984 7fe6cccb4c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:05.984 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:05.984 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:05.985 7fe6cccb4c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:09.985 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:09.986 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:09.987 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:09.988 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:09.988 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:09.988 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:09.988 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:10.479 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4715-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762999150479456, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762999136000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762999136000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762999136000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2025-11-13 00:53:20.682783", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12697-1.novalocal", "build-id": "v0+nightly", "build-number": "202511130009"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/13/2025 01:59:10.479 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13779460, partition_space_available_1k=67311024, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1291612, cpu_share=0.0, mem_res=1280956, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52412, cpu_share=0.0, mem_res=50104, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54752, cpu_share=0.0, mem_res=44872, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=5719156, free=13119680, buffers=610300, cached=13758616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.33, five_min_avg=1.68, fifteen_min_avg=1.13, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 01:59:10.480 7fe6cccd39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:10.480 7fe6cccd39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762999136000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762999136000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762999136000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-11-13 00:53:20.682783", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-12697-1.novalocal", "build-id": "v0+nightly", "build-number": "202511130009"}]} >> 11/13/2025 01:59:10.480 7fe6cccd39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:10.480 7fe6cccd39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:10.480 7fe6cccd39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13779460 partition_space_available_1k = 67311024 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1291612 cpu_share = 0.0 mem_res = 1280956 >>zookeeper : << mem_virt = 52412 cpu_share = 0.0 mem_res = 50104 >>contrail-config-database-nodemgr : << mem_virt = 54752 cpu_share = 0.0 mem_res = 44872 >> } system_mem_usage = << total = 32665116 used = 5719156 free = 13119680 buffers = 610300 cached = 13758616 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.33 five_min_avg = 1.68 fifteen_min_avg = 1.13 cpu_share = 0 node_type = config-database-node >> >> 11/13/2025 01:59:10.480 7fe6cccd39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:13.987 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:13.988 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:13.988 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:13.989 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:13.989 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:13.989 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:13.989 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:13.989 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:13.990 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:13.990 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:13.990 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:13.990 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:13.990 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:13.990 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:17.992 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:17.993 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:17.993 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:17.993 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:17.993 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:17.994 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:17.994 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:17.994 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:17.994 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:17.994 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:17.995 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:17.995 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:17.995 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:17.995 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:21.995 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:21.996 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:21.997 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:21.997 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:21.997 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:21.997 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:21.997 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:21.998 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:25.999 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:26.000 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:26.000 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:26.000 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:26.001 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:26.002 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:30.002 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:30.003 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:30.003 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:30.003 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:30.003 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:30.003 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:30.003 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:30.004 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:34.007 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:34.008 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:34.008 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:34.008 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:34.008 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:34.009 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:38.011 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:38.012 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:38.013 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:38.014 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:38.014 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:38.014 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:38.014 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:38.014 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:42.015 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:42.016 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:42.016 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:42.016 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:42.016 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:42.016 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:42.017 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:46.018 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:46.019 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:46.019 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:46.019 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:46.019 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:46.020 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:46.021 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:46.022 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:50.019 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:50.023 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:50.024 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:50.024 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:50.024 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:50.024 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:50.024 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:50.025 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:54.027 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:54.028 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:54.028 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:54.028 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:54.029 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:54.029 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:54.029 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:54.029 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:54.029 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:54.029 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:54.030 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:54.030 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:54.030 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:54.030 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:58.031 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 01:59:58.032 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 01:59:58.032 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:58.033 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:58.034 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 01:59:58.034 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:58.312 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 01:59:58.578 7fe6cccd37c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:58.578 7fe6cccd37c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. database_usage = [ << disk_space_used_1k = 14693460 disk_space_available_1k = 66397024 config_db_size_1k = 1028 >>, ] >> 11/13/2025 01:59:58.578 7fe6cccd37c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:59.097 7fe6cf9cc940 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4715-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762999199097803, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.57:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/13/2025 01:59:59.099 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:59.099 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 01:59:59.099 7fe6cc9c7ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 01:59:59.966 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:00:02.035 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:02.036 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:02.037 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:02.037 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:02.037 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:02.037 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:02.037 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:02.038 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:02.038 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:02.038 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:02.038 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:02.038 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:02.038 7fe6cccb4860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:02.039 7fe6cccb4860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:05.002 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14693488, partition_space_available_1k=66396996, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1335648, cpu_share=0.29, mem_res=1322012, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=68348, cpu_share=0.02, mem_res=65192, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54852, cpu_share=0.01, mem_res=44872, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=6611768, free=11478252, buffers=651168, cached=14467256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=1.79, fifteen_min_avg=1.19, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:00:05.002 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:05.003 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14693488 partition_space_available_1k = 66396996 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1335648 cpu_share = 0.29 mem_res = 1322012 >>zookeeper : << mem_virt = 68348 cpu_share = 0.02 mem_res = 65192 >>contrail-config-database-nodemgr : << mem_virt = 54852 cpu_share = 0.01 mem_res = 44872 >> } system_mem_usage = << total = 32665116 used = 6611768 free = 11478252 buffers = 651168 cached = 14467256 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.35 five_min_avg = 1.79 fifteen_min_avg = 1.19 cpu_share = 3.12 node_type = config-database-node >> >> 11/13/2025 02:00:05.003 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:06.038 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:06.039 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:06.039 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:06.039 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:06.040 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:06.041 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:10.043 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:10.044 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:10.044 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:10.045 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:10.046 7fe6cc9c7e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:10.046 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:14.045 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:14.046 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:14.046 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:14.047 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:14.048 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:14.048 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:14.049 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:14.049 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:18.051 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:18.052 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:18.053 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:18.053 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:18.053 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:18.053 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:18.053 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:18.054 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:18.054 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:18.054 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:18.054 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:18.054 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:18.055 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:18.055 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:22.055 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:22.056 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:22.056 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:22.056 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:22.057 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:26.061 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:26.062 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:26.062 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:26.062 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:26.063 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:26.063 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:26.063 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:26.063 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:26.064 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:26.064 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:26.064 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:26.064 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:26.064 7fe6cccd3680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:26.064 7fe6cccd3680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:30.063 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:30.064 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:30.064 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:30.064 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:30.065 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:30.066 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:30.066 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:34.069 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:34.070 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:34.070 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:34.070 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:34.070 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:34.071 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:34.072 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:38.074 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:38.075 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:38.075 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:38.075 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:38.075 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:38.076 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:38.076 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:38.076 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:38.076 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:38.076 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:38.077 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:38.077 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:38.077 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:38.077 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:42.079 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:42.080 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:42.080 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:42.080 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:42.080 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:42.081 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:42.081 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:42.081 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:42.082 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:42.082 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:42.082 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:42.082 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:42.082 7fe6cccd3400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:42.082 7fe6cccd3400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:46.083 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:46.084 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:46.084 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:46.084 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:46.084 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:46.084 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:46.085 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:46.085 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:46.085 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:46.085 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:46.085 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:46.085 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:46.086 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:46.086 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:50.086 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:50.087 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:50.088 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:50.089 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:50.089 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:50.089 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:50.089 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:54.091 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:54.092 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:54.092 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:54.092 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:54.092 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:54.092 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:54.093 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:58.096 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:00:58.097 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:00:58.098 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:00:58.098 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:58.098 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:00:58.098 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:00:58.098 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:00:58.098 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:00:58.099 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:58.099 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:00:58.099 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:58.099 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:58.099 7fe6cccd3c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:00:58.099 7fe6cccd3c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:58.324 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:00:58.635 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:00:58.635 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. database_usage = [ << disk_space_used_1k = 16425296 disk_space_available_1k = 64665188 config_db_size_1k = 1028 >>, ] >> 11/13/2025 02:00:58.635 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:00.067 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:01:02.099 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:02.101 7fe6cccb4900 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:02.101 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:02.101 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:02.101 7fe6cccb4900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:02.101 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:02.102 7fe6cccb4900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:05.099 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16426384, partition_space_available_1k=64664100, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1322736, cpu_share=0.05, mem_res=1309404, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=69224, cpu_share=0.0, mem_res=66060, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54944, cpu_share=0.01, mem_res=44872, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=7625560, free=8596180, buffers=656460, cached=16259424, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=1.83, fifteen_min_avg=1.24, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:01:05.100 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:05.100 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16426384 partition_space_available_1k = 64664100 percentage_partition_space_used = 20 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1322736 cpu_share = 0.05 mem_res = 1309404 >>zookeeper : << mem_virt = 69224 cpu_share = 0.0 mem_res = 66060 >>contrail-config-database-nodemgr : << mem_virt = 54944 cpu_share = 0.01 mem_res = 44872 >> } system_mem_usage = << total = 32665116 used = 7625560 free = 8596180 buffers = 656460 cached = 16259424 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.14 five_min_avg = 1.83 fifteen_min_avg = 1.24 cpu_share = 2.62 node_type = config-database-node >> >> 11/13/2025 02:01:05.100 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:06.102 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:06.103 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:06.103 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:06.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:06.105 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:06.105 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:06.105 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:06.105 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:10.104 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:10.105 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:10.106 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:10.106 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:10.106 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:10.107 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:10.107 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:10.107 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:10.107 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:10.107 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:10.108 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:10.108 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:10.108 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:10.108 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:14.107 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:14.108 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:14.108 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:14.109 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:14.109 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:14.109 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:14.109 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:14.109 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:14.110 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:14.110 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:14.110 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:14.110 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:14.110 7fe6cccd35e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:14.110 7fe6cccd35e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:18.111 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:18.112 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:18.113 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:18.114 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:18.114 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:18.114 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:18.114 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:18.114 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:22.115 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:22.116 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:22.116 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:22.116 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:22.117 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:22.118 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:26.117 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:26.118 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:26.118 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:26.118 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:26.119 7fe6cccd3040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:30.119 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:30.120 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/13/2025 02:01:30.120 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:30.121 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:30.122 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:30.122 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:34.123 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:34.124 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/13/2025 02:01:34.125 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/13/2025 02:01:34.126 7fe6cccd3a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/13/2025 02:01:45.288 7fe6cc9c7ea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 11/13/2025 02:01:45.289 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 11/13/2025 02:01:45.289 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 11/13/2025 02:01:45.289 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:45.289 7fe6cd7f14a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 11/13/2025 02:01:45.290 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:45.290 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:45.290 7fe6cd7f14a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8112 start_time = 1762999137980083 collector_name = collector_ip = 10.0.0.57:8086 collector_list = [ 10.0.0.57:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/13/2025 02:01:45.291 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/13/2025 02:01:49.291 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/13/2025 02:01:49.292 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4715-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.57:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 11/13/2025 02:01:49.293 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/13/2025 02:01:49.294 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/13/2025 02:01:49.294 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/13/2025 02:01:49.301 7fe6cccb4680 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 11/13/2025 02:01:49.302 7fe6cccb4680 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/13/2025 02:01:49.302 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/13/2025 02:01:49.302 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 11/13/2025 02:01:49.302 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/13/2025 02:01:49.302 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/13/2025 02:01:49.302 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/13/2025 02:01:49.303 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/13/2025 02:01:49.303 7fe6cccd3900 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/13/2025 02:01:49.304 7fe6cccd3900 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/13/2025 02:01:55.311 7fe6cccb4680 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438 11/13/2025 02:01:55.312 7fe6cccb4680 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/13/2025 02:01:55.312 7fe6cc9c7e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 11/13/2025 02:01:55.312 7fe6cc9c7e00 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 11/13/2025 02:01:55.312 7fe6cc9c7e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/13/2025 02:01:55.312 7fe6cc9c7e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/13/2025 02:01:55.312 7fe6cc9c7e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/13/2025 02:01:55.313 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/13/2025 02:01:55.313 7fe6cd7f14a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/13/2025 02:01:58.384 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:02:00.242 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:02:05.278 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17649048, partition_space_available_1k=63441436, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1325352, cpu_share=0.05, mem_res=1311996, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=136004, cpu_share=0.04, mem_res=132012, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54980, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9474360, free=5376692, buffers=684100, cached=17603032, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.22, five_min_avg=2.42, fifteen_min_avg=1.48, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:02:58.384 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:03:00.634 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:03:05.673 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19560612, partition_space_available_1k=61529872, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1323564, cpu_share=0.05, mem_res=1296308, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146352, cpu_share=0.01, mem_res=142940, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=47176, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9408904, free=18383492, buffers=701124, cached=4644772, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=2.77, fifteen_min_avg=1.67, cpu_share=5.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:03:40.714 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/13/2025 02:03:42.818 7fe6cd7f14a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/13/2025 02:03:58.385 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:04:00.425 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:04:05.504 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19770252, partition_space_available_1k=61320232, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327536, cpu_share=0.05, mem_res=1304284, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146788, cpu_share=0.0, mem_res=143372, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50400, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9985648, free=17478796, buffers=710408, cached=4969168, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.04, five_min_avg=2.76, fifteen_min_avg=1.74, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:04:58.423 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:05:00.377 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:05:05.412 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21053144, partition_space_available_1k=60037340, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344516, cpu_share=0.07, mem_res=1320580, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147724, cpu_share=0.0, mem_res=144304, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50412, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10586244, free=15449780, buffers=744956, cached=6357372, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=3.07, fifteen_min_avg=1.92, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:05:58.464 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:06:00.381 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:06:05.418 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21086824, partition_space_available_1k=60003660, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342016, cpu_share=0.2, mem_res=1317832, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=163880, cpu_share=0.02, mem_res=160348, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50412, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10626836, free=15358396, buffers=745660, cached=6406368, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.23, five_min_avg=3.33, fifteen_min_avg=2.08, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:06:58.412 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:07:00.280 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:07:05.318 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21197880, partition_space_available_1k=59892604, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339512, cpu_share=0.09, mem_res=1315092, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166356, cpu_share=0.0, mem_res=162804, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50436, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11296352, free=14565400, buffers=745764, cached=6529596, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=3.35, fifteen_min_avg=2.17, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:07:58.430 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:08:00.243 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:08:05.282 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21229348, partition_space_available_1k=59861136, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339456, cpu_share=0.15, mem_res=1314608, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=172040, cpu_share=0.01, mem_res=168428, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50424, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11036652, free=14772716, buffers=746172, cached=6581492, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=3.4, fifteen_min_avg=2.26, cpu_share=4.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:08:58.468 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:09:00.326 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:09:05.372 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21257676, partition_space_available_1k=59832808, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1360336, cpu_share=0.1, mem_res=1335364, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184516, cpu_share=0.02, mem_res=180860, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50424, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10868208, free=14915592, buffers=746412, cached=6607032, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.43, fifteen_min_avg=2.34, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:09:58.426 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:10:00.145 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:10:05.181 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21348048, partition_space_available_1k=59742436, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350952, cpu_share=0.09, mem_res=1325264, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=180348, cpu_share=0.0, mem_res=176680, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50424, cpu_share=0.01, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=12001932, free=13734228, buffers=746476, cached=6654796, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.58, five_min_avg=4.24, fifteen_min_avg=2.7, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:10:58.467 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:11:00.222 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:11:05.254 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21344260, partition_space_available_1k=59746224, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364160, cpu_share=0.07, mem_res=1338272, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=182496, cpu_share=0.0, mem_res=178812, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50432, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10988504, free=14718268, buffers=746536, cached=6683148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.58, five_min_avg=4.23, fifteen_min_avg=2.8, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:11:58.438 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:12:00.209 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:12:05.239 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21368228, partition_space_available_1k=59722256, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363192, cpu_share=0.08, mem_res=1337232, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184884, cpu_share=0.0, mem_res=181180, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50436, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11298488, free=14379624, buffers=746580, cached=6711524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.98, five_min_avg=4.1, fifteen_min_avg=2.84, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:12:58.500 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:13:00.387 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:13:05.422 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21479596, partition_space_available_1k=59610888, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364156, cpu_share=0.07, mem_res=1338120, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=186912, cpu_share=0.0, mem_res=183188, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50440, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11006144, free=14556220, buffers=746652, cached=6827232, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.43, five_min_avg=4.25, fifteen_min_avg=2.98, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:13:58.433 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:14:00.098 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:14:05.143 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21489924, partition_space_available_1k=59600560, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364512, cpu_share=0.05, mem_res=1338404, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187512, cpu_share=0.0, mem_res=183788, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50444, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11039536, free=14512288, buffers=746656, cached=6837536, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.33, five_min_avg=4.05, fifteen_min_avg=2.99, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:14:58.454 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/13/2025 02:15:00.221 7fe6cf9cc940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/13/2025 02:15:05.251 7fe6cf9cc940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4715-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21501036, partition_space_available_1k=59589448, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338832, cpu_share=0.08, mem_res=1312072, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189832, cpu_share=0.0, mem_res=185952, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50444, cpu_share=0.02, mem_res=44868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10974772, free=14563016, buffers=746704, cached=6850572, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.39, five_min_avg=4.8, fifteen_min_avg=3.3, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/13/2025 02:15:50.359 7fe6cc9c7d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map