11/10/2025 01:50:08.520 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/10/2025 01:50:08.521 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/10/2025 01:50:08.521 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/10/2025 01:50:08.530 7fcd3d120400 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 11/10/2025 01:50:08.530 7fcd3d120400 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 11/10/2025 01:50:08.531 7fcd3d1207c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 11/10/2025 01:50:08.532 7fcd3d1207c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/10/2025 01:50:08.532 7fcd3d1207c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/10/2025 01:50:08.532 7fcd3d1207c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/10/2025 01:50:08.532 7fcd3d1207c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.533 7fcd3d1207c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/10/2025 01:50:08.533 7fcd3d1207c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.533 7fcd3d1207c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.533 7fcd3d1207c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/10/2025 01:50:08.533 7fcd3d1207c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.661 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 6185670155695825073790524613851539377720687926464420844699469489119400658160, 'start': '1762739406000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 111780841645895237236552737500616473455583310320071318230405981224546131656286, 'start': '1762739406000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 84538944721278722651619973702779243084569996502269274791973064569897139848966, 'start': '1762739406000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 11/10/2025 01:50:08.736 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.737 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-10 00:46:32.898846", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12558-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 11/10/2025 01:50:08.737 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.737 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4676-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762739408737881, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/10/2025 01:50:08.738 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4676-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762739408738183, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762739406000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762739406000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762739406000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-10 00:46:32.898846", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12558-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/10/2025 01:50:08.740 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.740 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/10/2025 01:50:08.740 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.740 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.740 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762739406000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762739406000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762739406000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-10 00:46:32.898846", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12558-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 11/10/2025 01:50:08.740 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:08.974 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:50:09.243 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/10/2025 01:50:09.245 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:09.245 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. database_usage = [ << disk_space_used_1k = 13678548 disk_space_available_1k = 67411936 config_db_size_1k = 108 >>, ] >> 11/10/2025 01:50:09.245 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:09.607 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 11/10/2025 01:50:09.607 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4676-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762739409607261, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/10/2025 01:50:09.608 7fcd3d2ff900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:09.609 7fcd3d2ff900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/10/2025 01:50:09.609 7fcd3d2ff900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:10.691 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:50:11.745 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 11/10/2025 01:50:12.532 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:12.533 7fcd3d2ffd60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:12.533 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:12.533 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:12.533 7fcd3d2ffd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:12.534 7fcd3d2ffd60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:12.758 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 11/10/2025 01:50:13.670 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 11/10/2025 01:50:14.518 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 11/10/2025 01:50:15.384 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 11/10/2025 01:50:16.284 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 11/10/2025 01:50:16.534 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:16.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:16.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:16.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:16.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:16.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:16.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:16.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:20.536 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:20.537 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:20.538 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:20.539 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:20.539 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:20.539 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:20.539 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:20.539 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:21.316 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4676-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762739421316682, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762739406000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762739406000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762739406000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-10 00:46:32.898846", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12558-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/10/2025 01:50:21.317 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13906340, partition_space_available_1k=67184144, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304036, cpu_share=0.0, mem_res=1293264, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52656, cpu_share=0.0, mem_res=50332, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52612, cpu_share=0.0, mem_res=42656, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=5356368, free=13266520, buffers=585936, cached=13998808, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.24, five_min_avg=1.38, fifteen_min_avg=0.98, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:50:21.317 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:21.317 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762739406000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762739406000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762739406000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-10 00:46:32.898846", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12558-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 11/10/2025 01:50:21.317 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:21.317 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:21.317 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13906340 partition_space_available_1k = 67184144 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1304036 cpu_share = 0.0 mem_res = 1293264 >>zookeeper : << mem_virt = 52656 cpu_share = 0.0 mem_res = 50332 >>contrail-config-database-nodemgr : << mem_virt = 52612 cpu_share = 0.0 mem_res = 42656 >> } system_mem_usage = << total = 32665120 used = 5356368 free = 13266520 buffers = 585936 cached = 13998808 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.24 five_min_avg = 1.38 fifteen_min_avg = 0.98 cpu_share = 0 node_type = config-database-node >> >> 11/10/2025 01:50:21.318 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:24.538 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:24.540 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:24.540 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:24.540 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:24.540 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:24.540 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:24.540 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:24.541 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:28.544 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:28.546 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:28.546 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:28.546 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:28.546 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:28.546 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:28.546 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:28.547 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:32.548 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:32.550 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:32.550 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:32.550 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:32.550 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:32.550 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:32.550 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:32.551 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:36.552 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:36.554 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:36.554 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:36.554 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:36.554 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:36.554 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:36.554 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:36.555 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:40.556 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:40.557 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:40.557 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:40.557 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:40.557 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:40.557 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:40.557 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:40.558 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:40.558 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:40.558 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:40.558 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:40.558 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:40.559 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:40.559 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:44.558 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:44.559 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:44.559 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:44.559 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:44.559 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:44.559 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:44.559 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:44.560 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:44.560 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:44.560 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:44.560 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:44.560 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:44.560 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:44.561 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:48.560 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:48.562 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:48.562 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:48.562 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:48.562 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:48.562 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:48.562 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:48.563 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:52.566 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:52.567 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:52.568 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:52.568 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:52.568 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:52.568 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:52.568 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:52.569 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:56.569 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:50:56.570 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:50:56.570 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:50:56.570 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:56.570 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:50:56.571 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:50:56.572 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:00.572 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:00.576 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:00.576 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:00.576 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:00.576 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:00.576 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:00.576 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:00.577 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:04.580 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:04.581 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:04.582 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:04.582 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:04.582 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:04.582 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:04.582 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:04.583 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:08.584 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:08.586 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:08.586 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:08.586 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:08.586 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:08.586 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:08.587 7fcd3d120cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:08.588 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:08.954 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:51:09.232 7fcd3d2ff7c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:09.232 7fcd3d2ff7c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. database_usage = [ << disk_space_used_1k = 14815124 disk_space_available_1k = 66275360 config_db_size_1k = 1028 >>, ] >> 11/10/2025 01:51:09.233 7fcd3d2ff7c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:09.747 7fcd3fffda00 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4676-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762739469747735, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.141:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/10/2025 01:51:09.751 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:09.751 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:09.751 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:10.589 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:51:12.587 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:12.589 7fcd3d2e42c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:12.589 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:12.589 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:12.590 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:12.591 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:12.591 7fcd3d2e42c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:12.591 7fcd3d2e42c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:15.623 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14815176, partition_space_available_1k=66275308, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1325040, cpu_share=0.32, mem_res=1311388, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75800, cpu_share=0.02, mem_res=72632, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52716, cpu_share=0.01, mem_res=42656, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6293016, free=11583752, buffers=626324, cached=14705036, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=1.6, fifteen_min_avg=1.08, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:51:15.623 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:15.624 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14815176 partition_space_available_1k = 66275308 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1325040 cpu_share = 0.32 mem_res = 1311388 >>zookeeper : << mem_virt = 75800 cpu_share = 0.02 mem_res = 72632 >>contrail-config-database-nodemgr : << mem_virt = 52716 cpu_share = 0.01 mem_res = 42656 >> } system_mem_usage = << total = 32665120 used = 6293016 free = 11583752 buffers = 626324 cached = 14705036 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.56 five_min_avg = 1.6 fifteen_min_avg = 1.08 cpu_share = 3.5 node_type = config-database-node >> >> 11/10/2025 01:51:15.624 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:16.591 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:16.592 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:16.592 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:16.593 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:16.594 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:16.594 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:16.594 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:20.597 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:20.598 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:20.599 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:20.599 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:20.599 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:20.599 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:20.599 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:20.599 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:20.600 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:20.600 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:20.600 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:20.600 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:20.600 7fcd3d120c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:20.600 7fcd3d120c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:24.599 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:24.601 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:24.602 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:24.602 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:24.602 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:24.602 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:24.602 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:28.605 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:28.606 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:28.607 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:28.607 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:28.607 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:28.607 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:28.607 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:28.607 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:28.608 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:28.608 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:28.608 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:28.608 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:28.608 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:28.608 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:32.610 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:32.611 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:32.611 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:32.612 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:32.612 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:32.612 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:32.612 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:32.612 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:32.613 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:32.613 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:32.613 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:32.613 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:32.613 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:32.613 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:36.616 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:36.618 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:36.619 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:36.619 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:36.619 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:36.619 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:36.619 7fcd3d2ff540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:36.619 7fcd3d2ff540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:40.620 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:40.621 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:40.621 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:40.621 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:40.621 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:40.621 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:40.622 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:40.623 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:44.624 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:44.626 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:44.626 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:44.626 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:44.626 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:44.626 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:44.626 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:44.627 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:48.630 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:48.631 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:48.631 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:48.631 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:48.632 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:48.633 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:48.633 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:52.633 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:52.634 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:52.635 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:52.635 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:52.635 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:52.635 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:52.635 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:52.636 7fcd3d120ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:56.637 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:51:56.639 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:51:56.639 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:51:56.639 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:56.639 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:51:56.639 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:51:56.639 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:51:56.640 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:00.641 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:00.642 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:00.643 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:00.644 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:00.644 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:00.644 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:00.644 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:00.644 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:04.646 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:04.648 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:04.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:04.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:04.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:04.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:04.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:04.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:08.649 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:08.651 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:08.651 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:08.651 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:08.651 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:08.651 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:08.651 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:08.652 7fcd3d2ff180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:08.940 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:52:09.209 7fcd3d2ff400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:09.210 7fcd3d2ff400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. database_usage = [ << disk_space_used_1k = 16548272 disk_space_available_1k = 64542212 config_db_size_1k = 1028 >>, ] >> 11/10/2025 01:52:09.210 7fcd3d2ff400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:10.570 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:52:12.652 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:12.654 7fcd3d2e4c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:12.654 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:12.654 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:12.654 7fcd3d2e4c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:12.655 7fcd3d2e4c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:12.656 7fcd3d2e4c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:15.604 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16548432, partition_space_available_1k=64542052, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1329608, cpu_share=0.06, mem_res=1316340, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=74796, cpu_share=0.0, mem_res=71624, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52808, cpu_share=0.01, mem_res=42656, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6535852, free=9472152, buffers=631040, cached=16496216, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=1.73, fifteen_min_avg=1.16, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:52:15.605 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:15.605 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16548432 partition_space_available_1k = 64542052 percentage_partition_space_used = 20 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1329608 cpu_share = 0.06 mem_res = 1316340 >>zookeeper : << mem_virt = 74796 cpu_share = 0.0 mem_res = 71624 >>contrail-config-database-nodemgr : << mem_virt = 52808 cpu_share = 0.01 mem_res = 42656 >> } system_mem_usage = << total = 32665120 used = 6535852 free = 9472152 buffers = 631040 cached = 16496216 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.35 five_min_avg = 1.73 fifteen_min_avg = 1.16 cpu_share = 2.62 node_type = config-database-node >> >> 11/10/2025 01:52:15.605 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:16.655 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:16.656 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:16.656 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:16.656 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:16.656 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:16.657 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:20.660 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:20.661 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:20.662 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:20.663 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:20.663 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:20.663 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:20.663 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:20.663 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:24.666 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:24.667 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:24.668 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:24.669 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:24.669 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:24.669 7fcd3d120e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:24.669 7fcd3d120e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:28.668 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:28.670 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:28.670 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:28.670 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:28.670 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:28.670 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:28.670 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:28.671 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:28.671 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:28.671 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:28.671 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:28.671 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:28.671 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:28.672 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:32.672 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:32.673 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:32.673 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:32.673 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:32.674 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:32.674 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:32.674 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:32.674 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:32.674 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:32.674 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:32.675 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:32.675 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:32.675 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:32.675 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:36.676 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:36.677 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:36.678 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:36.679 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:36.679 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:36.679 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:36.679 7fcd3d2ff720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:36.679 7fcd3d2ff720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:40.679 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:40.680 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:40.680 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:40.680 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:40.680 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:40.680 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:40.680 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:40.681 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:40.681 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:40.681 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:40.681 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:40.681 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:40.681 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:40.682 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:44.684 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:44.686 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:44.686 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:44.686 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:44.686 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:44.686 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:44.686 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:44.687 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:44.687 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:44.687 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:44.687 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:44.687 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:44.688 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:44.688 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:48.688 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:48.690 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:48.691 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:48.691 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:48.691 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:48.691 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:48.691 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:48.691 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:52.694 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:52.695 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/10/2025 01:52:52.696 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:52.696 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:52.696 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:52.696 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:52.696 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/10/2025 01:52:52.696 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/10/2025 01:52:52.697 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:52.697 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/10/2025 01:52:52.697 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:52.697 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:52.697 7fcd3d2e4e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:52:52.697 7fcd3d2e4e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:52:56.696 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/10/2025 01:52:56.698 7fcd3d2e47c0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 11/10/2025 01:52:56.699 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/10/2025 01:52:56.699 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/10/2025 01:52:56.699 7fcd3d2e47c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/10/2025 01:53:06.156 7fcd3d120b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 11/10/2025 01:53:06.156 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 11/10/2025 01:53:06.156 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 11/10/2025 01:53:06.157 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:53:06.157 7fcd3dd3c540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 11/10/2025 01:53:06.158 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:53:06.158 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:53:06.158 7fcd3dd3c540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8112 start_time = 1762739408532485 collector_name = collector_ip = 10.0.0.141:8086 collector_list = [ 10.0.0.141:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/10/2025 01:53:06.158 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:53:08.961 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:53:09.269 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:53:09.269 7fcd3dd3c540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. database_usage = [ << disk_space_used_1k = 17381576 disk_space_available_1k = 63708908 config_db_size_1k = 1028 >>, ] >> 11/10/2025 01:53:09.269 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/10/2025 01:53:10.157 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4676-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.141:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/10/2025 01:53:10.159 7fcd3dd3c540 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 11/10/2025 01:53:10.160 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/10/2025 01:53:10.160 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/10/2025 01:53:10.160 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/10/2025 01:53:10.167 7fcd3d2e45e0 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 11/10/2025 01:53:10.167 7fcd3d2e45e0 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/10/2025 01:53:10.168 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/10/2025 01:53:10.168 7fcd3dd3c540 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 11/10/2025 01:53:10.168 7fcd3dd3c540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/10/2025 01:53:10.168 7fcd3dd3c540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/10/2025 01:53:10.168 7fcd3dd3c540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/10/2025 01:53:10.169 7fcd3dd3c540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/10/2025 01:53:10.169 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/10/2025 01:53:10.170 7fcd3d120cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/10/2025 01:53:11.082 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:53:16.121 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17493672, partition_space_available_1k=63596812, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1323948, cpu_share=0.05, mem_res=1310672, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77408, cpu_share=0.0, mem_res=74216, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52852, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8880068, free=5801108, buffers=650092, cached=17805224, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.35, five_min_avg=2.32, fifteen_min_avg=1.4, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:53:16.166 7fcd3d2e45e0 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438 11/10/2025 01:53:16.167 7fcd3d2e45e0 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/10/2025 01:53:16.167 7fcd3d2e4540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 11/10/2025 01:53:16.172 7fcd3d2e4540 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 11/10/2025 01:53:16.173 7fcd3d2e4540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/10/2025 01:53:16.173 7fcd3d2e4540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/10/2025 01:53:16.173 7fcd3d2e4540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/10/2025 01:53:16.173 7fcd3ddecf40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/10/2025 01:53:16.174 7fcd3ddecf40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/10/2025 01:54:09.176 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:54:11.404 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:54:16.459 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19606632, partition_space_available_1k=61483852, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1315944, cpu_share=0.06, mem_res=1306416, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=128028, cpu_share=0.05, mem_res=124496, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44900, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9285736, free=18446240, buffers=675660, cached=4728840, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.41, five_min_avg=2.77, fifteen_min_avg=1.62, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:55:09.005 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:55:10.951 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:55:16.025 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19625100, partition_space_available_1k=61465384, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1335700, cpu_share=0.04, mem_res=1308500, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=127916, cpu_share=0.0, mem_res=124512, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48196, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9275572, free=18375532, buffers=678792, cached=4806560, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=2.64, fifteen_min_avg=1.65, cpu_share=3.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:56:01.687 7fcd3d120860 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/10/2025 01:56:09.031 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:56:11.099 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:56:14.284 7fcd3d2e4540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/10/2025 01:56:16.139 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19813268, partition_space_available_1k=61277216, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333108, cpu_share=0.06, mem_res=1305276, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=127924, cpu_share=0.0, mem_res=124516, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44948, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10087880, free=17378792, buffers=684720, cached=4985140, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=2.52, fifteen_min_avg=1.67, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:56:16.706 7fcd3ddecf40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/10/2025 01:57:09.014 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:57:10.901 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:57:15.940 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20087188, partition_space_available_1k=61003296, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334736, cpu_share=0.05, mem_res=1306836, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=127948, cpu_share=0.0, mem_res=124536, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44952, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10034556, free=16752908, buffers=689540, cached=5658568, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.66, five_min_avg=2.52, fifteen_min_avg=1.72, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:58:09.071 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:58:11.064 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:58:16.103 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20204916, partition_space_available_1k=60885568, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364188, cpu_share=0.34, mem_res=1335612, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148524, cpu_share=0.03, mem_res=144964, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44956, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10266348, free=16392160, buffers=690888, cached=5785940, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=2.84, fifteen_min_avg=1.88, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:59:09.032 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/10/2025 01:59:10.859 7fcd3fffda00 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 11/10/2025 01:59:15.895 7fcd3fffda00 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4676-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20252816, partition_space_available_1k=60837668, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353848, cpu_share=0.36, mem_res=1323896, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=151624, cpu_share=0.03, mem_res=147860, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44956, cpu_share=0.02, mem_res=42652, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10409028, free=16197984, buffers=690956, cached=5837300, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.09, five_min_avg=4.09, fifteen_min_avg=2.36, cpu_share=5.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/10/2025 01:59:46.700 7fcd3d120ea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map