10/31/2024 02:17:29.705 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/31/2024 02:17:29.705 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/31/2024 02:17:29.705 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/31/2024 02:17:29.712 7fdabfb4d248 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 10/31/2024 02:17:29.713 7fdabfb4d248 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 10/31/2024 02:17:29.713 7fdabfb4d448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/31/2024 02:17:29.714 7fdabfb4d448 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/31/2024 02:17:29.714 7fdabfb4d448 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/31/2024 02:17:29.715 7fdabfb4d448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.846 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 97744613167006267662940960509328238560132910806995256876612990510268995496424, 'start': '1730341047000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 67056768000904608110452588989009171283301034492978045108924215552725674725077, 'start': '1730341047000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 60890816254487764576950876962651143912621510730474534504031503012621574542512, 'start': '1730341048000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/31/2024 02:17:29.985 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.985 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-31 01:12:39.614478", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6264-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 10/31/2024 02:17:29.985 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.986 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2183-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730341049986295, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/31/2024 02:17:29.986 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2183-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730341049986612, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730341047000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730341047000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730341048000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-31 01:12:39.614478", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6264-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/31/2024 02:17:29.989 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.990 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/31/2024 02:17:29.990 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.990 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:29.990 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730341047000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730341047000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730341048000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-31 01:12:39.614478", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6264-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/31/2024 02:17:29.990 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:30.371 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:17:30.721 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/31/2024 02:17:30.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:30.724 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. database_usage = [ << disk_space_used_1k = 13530516 disk_space_available_1k = 67559968 config_db_size_1k = 108 >>, ] >> 10/31/2024 02:17:30.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:31.161 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 10/31/2024 02:17:31.161 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2183-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730341051161292, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/31/2024 02:17:31.164 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:31.164 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/31/2024 02:17:31.164 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:32.249 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:17:33.575 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 10/31/2024 02:17:33.714 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:33.716 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:33.716 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:33.716 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:33.716 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:33.716 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:33.717 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:33.718 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:34.471 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 10/31/2024 02:17:35.360 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 10/31/2024 02:17:36.271 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 10/31/2024 02:17:37.211 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 10/31/2024 02:17:37.717 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:37.718 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:37.718 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:37.719 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:37.720 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:37.720 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:37.720 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:37.720 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:38.059 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 10/31/2024 02:17:41.719 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:41.721 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:41.721 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:41.721 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:41.722 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:41.722 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:41.722 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:41.722 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:41.722 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:41.722 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:41.723 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:41.723 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:41.723 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:41.723 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:43.085 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2183-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730341063085412, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730341047000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730341047000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730341048000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-31 01:12:39.614478", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6264-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/31/2024 02:17:43.085 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13841496, partition_space_available_1k=67248988, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1219172, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50376, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53416, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=4769408, free=13581520, buffers=578700, cached=13735492, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.05, five_min_avg=1.54, fifteen_min_avg=1.1, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:17:43.086 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:43.086 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730341047000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730341047000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730341048000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-31 01:12:39.614478", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6264-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/31/2024 02:17:43.086 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:43.086 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:43.086 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13841496 partition_space_available_1k = 67248988 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1219172 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 50376 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 53416 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665120 used = 4769408 free = 13581520 buffers = 578700 cached = 13735492 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.05 five_min_avg = 1.54 fifteen_min_avg = 1.1 cpu_share = 0 node_type = config-database-node >> >> 10/31/2024 02:17:43.086 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:45.722 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:45.723 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:45.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:45.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:45.724 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:45.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:45.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:45.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:45.725 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:45.725 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:45.725 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:45.725 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:45.725 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:45.725 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:49.724 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:49.725 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:49.726 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:49.726 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:49.726 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:49.726 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:49.726 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:49.726 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:49.727 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:49.727 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:49.727 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:49.727 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:49.727 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:49.727 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:53.728 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:53.757 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:53.758 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:53.758 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:53.758 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:53.758 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:53.758 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:53.759 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:57.760 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:17:57.762 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:17:57.762 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:17:57.762 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:57.762 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:17:57.763 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:17:57.763 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:17:57.763 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:17:57.763 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:57.763 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:17:57.763 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:57.764 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:17:57.764 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:17:57.764 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:01.763 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:01.772 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:01.773 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:01.773 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:01.774 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:01.774 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:01.774 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:01.774 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:01.774 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:01.774 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:01.775 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:01.775 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:01.775 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:01.775 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:05.776 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:05.778 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:05.778 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:05.778 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:05.778 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:05.778 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:05.778 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:05.779 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:05.779 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:05.779 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:05.779 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:05.780 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:05.780 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:05.780 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:09.780 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:09.781 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:09.782 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:09.783 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:09.783 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:09.783 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:09.783 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:09.783 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:13.785 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:13.787 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:13.787 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:13.787 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:13.787 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:13.787 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:13.787 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:13.788 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:17.788 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:17.789 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:17.789 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:17.789 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:17.790 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:17.790 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:17.790 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:17.790 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:17.791 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:17.791 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:17.791 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:17.791 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:17.791 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:17.791 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:21.793 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:21.794 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:21.794 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:21.795 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:21.795 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:21.795 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:21.795 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:21.795 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:21.796 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:21.796 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:21.796 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:21.796 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:21.796 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:21.796 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:25.796 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:25.798 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:25.798 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:25.798 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:25.798 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:25.798 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:25.798 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:25.799 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:29.799 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:29.800 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:29.800 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:29.800 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:29.800 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:29.800 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:29.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:30.251 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:18:30.587 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:30.587 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. database_usage = [ << disk_space_used_1k = 14802872 disk_space_available_1k = 66287612 config_db_size_1k = 1460 >>, ] >> 10/31/2024 02:18:30.588 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:31.207 7fdac62b9f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2183-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730341111207223, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.31:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/31/2024 02:18:31.210 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:31.210 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:31.210 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:32.073 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:18:33.801 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:33.803 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:33.803 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:33.803 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:33.803 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:33.803 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:33.803 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:33.804 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:33.804 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:33.804 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:33.804 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:33.804 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:33.804 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:33.805 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:37.098 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14802892, partition_space_available_1k=66287592, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1899488, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50384, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53692, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=5343972, free=12162104, buffers=640900, cached=14518144, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.17, five_min_avg=1.69, fifteen_min_avg=1.18, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:18:37.098 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:37.098 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14802892 partition_space_available_1k = 66287592 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1899488 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 50384 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 53692 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665120 used = 5343972 free = 12162104 buffers = 640900 cached = 14518144 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.17 five_min_avg = 1.69 fifteen_min_avg = 1.18 cpu_share = 3.0 node_type = config-database-node >> >> 10/31/2024 02:18:37.098 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:37.803 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:37.805 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:37.805 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:37.805 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:37.805 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:37.805 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:37.805 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:37.806 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:41.809 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:41.810 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:41.810 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:41.810 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:41.810 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:41.810 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:41.810 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:41.811 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:45.811 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:45.812 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:45.812 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:45.812 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:45.812 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:45.812 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:45.812 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:45.813 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:49.812 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:49.813 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:49.814 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:49.814 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:49.814 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:49.814 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:49.814 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:49.814 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:49.815 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:49.815 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:49.815 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:49.815 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:49.815 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:49.815 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:53.816 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:53.817 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:53.818 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:53.818 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:53.818 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:53.818 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:53.818 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:53.819 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:57.820 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:18:57.821 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:18:57.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:57.823 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:18:57.823 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:57.823 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:18:57.823 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:18:57.823 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:01.822 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:01.823 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:01.823 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:01.823 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:01.823 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:01.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:01.825 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:01.825 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:05.824 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:05.825 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:05.826 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:05.826 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:05.826 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:05.826 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:05.826 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:05.827 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:09.828 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:09.829 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:09.829 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:09.829 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:09.830 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:09.830 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:09.830 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:09.830 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:09.830 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:09.831 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:09.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:09.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:09.831 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:09.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:13.830 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:13.831 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:13.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:13.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:13.831 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:13.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:13.831 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:13.832 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:17.832 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:17.833 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:17.833 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:17.833 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:17.834 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:17.835 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:17.835 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:21.837 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:21.838 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:21.838 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:21.838 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:21.838 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:21.838 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:21.838 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:21.839 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:25.841 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:25.842 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:25.843 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:25.843 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:25.843 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:25.843 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:25.843 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:25.843 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:25.844 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:25.844 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:25.844 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:25.844 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:25.844 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:25.844 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:29.845 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:29.846 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:29.846 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:29.846 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:29.846 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:29.846 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:29.846 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:29.847 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:30.297 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:19:30.666 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:30.666 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. database_usage = [ << disk_space_used_1k = 15105672 disk_space_available_1k = 65984812 config_db_size_1k = 1648 >>, ] >> 10/31/2024 02:19:30.666 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:32.110 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:19:33.846 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:33.848 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:33.848 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:33.848 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:33.848 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:33.848 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:33.849 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:33.850 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:37.136 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15106008, partition_space_available_1k=65984476, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1920260, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=79884, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54084, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6159600, free=10941532, buffers=668404, cached=14895584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.73, five_min_avg=1.94, fifteen_min_avg=1.3, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:19:37.137 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:37.137 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 15106008 partition_space_available_1k = 65984476 percentage_partition_space_used = 19 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1920260 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 79884 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 54084 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665120 used = 6159600 free = 10941532 buffers = 668404 cached = 14895584 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.73 five_min_avg = 1.94 fifteen_min_avg = 1.3 cpu_share = 3.75 node_type = config-database-node >> >> 10/31/2024 02:19:37.137 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:37.848 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:37.850 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:37.850 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:37.850 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:37.851 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:37.851 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:37.851 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:37.852 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:37.852 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:37.852 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:37.852 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:37.852 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:37.852 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:37.853 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:41.852 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:41.853 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:41.853 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:41.854 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:41.854 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:41.854 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:41.854 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:41.854 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:41.854 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:41.855 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:41.855 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:41.855 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:41.855 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:41.855 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:45.856 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:45.857 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:45.858 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:45.858 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:45.858 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:45.858 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:45.858 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:45.859 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:49.858 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:49.859 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:49.860 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:49.860 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:49.860 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:49.860 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:49.860 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:49.861 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:53.861 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:53.862 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:53.863 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:53.863 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:53.863 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:53.863 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:53.863 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:53.863 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:53.864 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:53.864 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:53.864 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:53.864 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:53.864 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:53.864 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:57.864 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:19:57.865 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:19:57.865 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:19:57.865 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:57.866 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:19:57.866 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:19:57.866 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:19:57.866 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:19:57.866 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:57.866 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:19:57.867 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:57.867 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:19:57.867 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:19:57.867 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:01.866 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:01.867 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:01.867 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:01.867 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:01.868 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:01.868 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:01.868 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:01.868 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:01.869 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:01.869 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:01.869 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:01.869 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:01.869 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:01.869 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:05.868 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:05.869 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:05.870 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:05.870 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:05.870 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:05.870 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:05.870 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:05.871 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:09.872 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:09.874 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:09.874 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:09.874 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:09.874 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:09.874 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:09.874 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:09.875 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:13.878 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:13.879 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:13.880 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:13.880 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:13.880 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:13.880 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:13.880 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:13.881 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:13.881 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:13.881 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:13.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:13.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:13.882 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:13.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:17.880 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:17.881 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:17.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:17.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:17.882 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:17.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:17.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:17.882 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:17.883 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:17.883 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:17.883 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:17.883 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:17.883 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:17.883 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:21.884 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:21.885 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:21.886 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:21.887 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:21.887 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:21.887 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:21.887 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:21.887 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:25.888 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:25.889 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:25.890 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:25.890 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:25.890 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:25.890 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:25.890 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:25.891 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:25.893 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:25.893 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:25.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:25.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:25.894 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:25.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:29.892 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:29.894 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:29.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:29.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:29.894 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:29.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:29.894 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:29.895 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:30.288 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:20:30.658 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:30.658 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. database_usage = [ << disk_space_used_1k = 15545228 disk_space_available_1k = 65545256 config_db_size_1k = 1612 >>, ] >> 10/31/2024 02:20:30.658 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:32.216 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:20:33.895 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:33.896 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:33.897 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:33.897 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:33.897 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:33.897 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:33.897 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:33.897 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:33.898 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:33.898 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:33.898 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:33.898 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:33.898 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:33.898 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:37.898 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:37.899 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:37.900 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:37.900 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:37.900 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:37.900 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:37.900 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:37.901 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:38.070 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15684496, partition_space_available_1k=65405988, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2023444, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80844, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54188, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6428200, free=9549180, buffers=692756, cached=15994984, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.14, five_min_avg=2.13, fifteen_min_avg=1.4, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:20:38.071 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:38.071 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 15684496 partition_space_available_1k = 65405988 percentage_partition_space_used = 19 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2023444 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 80844 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 54188 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665120 used = 6428200 free = 9549180 buffers = 692756 cached = 15994984 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.14 five_min_avg = 2.13 fifteen_min_avg = 1.4 cpu_share = 2.75 node_type = config-database-node >> >> 10/31/2024 02:20:38.071 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:41.901 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:41.902 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:41.903 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:41.903 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:41.903 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:41.903 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:41.903 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:41.903 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:41.904 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:41.904 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:41.904 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:41.904 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:41.904 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:41.904 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:45.906 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:45.908 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:45.908 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:45.908 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:45.908 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:45.908 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:45.908 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:45.909 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:49.908 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:49.909 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:49.909 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:49.910 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:49.910 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:49.910 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:49.910 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:49.910 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:49.911 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:49.911 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:49.911 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:49.911 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:49.911 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:49.911 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:53.913 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:53.914 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:53.914 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:53.915 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:53.915 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:53.915 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:53.915 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:53.916 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:57.916 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:20:57.917 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:20:57.918 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:20:57.918 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:57.918 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:20:57.918 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:20:57.918 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:20:57.919 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:01.921 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:01.922 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:01.922 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:01.922 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:01.922 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:01.922 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:01.923 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:01.924 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:05.923 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:05.924 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:05.924 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:05.924 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:05.925 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:05.926 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:05.926 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:05.926 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:09.928 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:09.929 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:09.930 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:09.931 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:09.931 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:09.931 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:09.931 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:09.931 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:13.933 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:13.935 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:13.935 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:13.935 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:13.935 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:13.935 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:13.935 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:13.936 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:13.936 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:13.936 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:13.936 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:13.936 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:13.936 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:13.937 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:17.936 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:17.937 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:17.937 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:17.937 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:17.937 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:17.937 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:17.937 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:17.938 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:21.940 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:21.941 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:21.942 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:21.942 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:21.942 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:21.942 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:21.942 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:21.943 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:25.944 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:25.945 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:25.946 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:25.946 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:25.946 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:25.946 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:25.946 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:25.946 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:25.947 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:25.947 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:25.947 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:25.947 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:25.947 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:25.947 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:29.948 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:29.950 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:29.950 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:29.950 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:29.951 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:29.951 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:29.951 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:29.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:29.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:29.952 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:29.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:29.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:29.953 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:29.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:30.310 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:21:30.714 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:30.715 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. database_usage = [ << disk_space_used_1k = 16085160 disk_space_available_1k = 65005324 config_db_size_1k = 1612 >>, ] >> 10/31/2024 02:21:30.715 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:32.271 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:21:33.950 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:33.952 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:33.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:33.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:33.952 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:33.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:33.952 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:33.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:37.867 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16085192, partition_space_available_1k=65005292, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2016716, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=81124, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54336, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7723016, free=8298848, buffers=697112, cached=15946144, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.58, five_min_avg=2.12, fifteen_min_avg=1.44, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:21:37.867 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:37.868 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16085192 partition_space_available_1k = 65005292 percentage_partition_space_used = 20 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2016716 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 81124 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 54336 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665120 used = 7723016 free = 8298848 buffers = 697112 cached = 15946144 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.58 five_min_avg = 2.12 fifteen_min_avg = 1.44 cpu_share = 2.88 node_type = config-database-node >> >> 10/31/2024 02:21:37.868 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:37.953 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:37.955 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:37.955 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:37.955 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:37.955 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:37.955 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:37.955 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:37.956 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:37.956 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:37.956 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:37.956 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:37.956 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:37.956 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:37.957 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:41.956 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:41.957 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:41.957 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:41.958 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:41.959 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:41.959 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:41.959 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:41.959 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:45.960 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:45.962 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:45.962 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:45.962 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:45.962 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:45.962 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:45.962 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:45.963 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:45.963 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:45.963 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:45.963 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:45.963 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:45.964 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:45.964 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:49.964 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:49.965 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:49.965 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:49.965 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:49.965 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:49.965 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:49.966 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:53.969 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:53.971 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:53.971 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:53.971 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:53.971 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:53.971 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:53.971 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:53.972 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:53.972 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:53.972 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:53.972 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:53.972 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:53.972 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:53.973 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:57.972 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:21:57.973 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:21:57.973 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:57.974 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:21:57.975 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:57.975 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:21:57.975 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:21:57.975 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:22:01.976 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:22:01.977 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/31/2024 02:22:01.977 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:22:01.978 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/31/2024 02:22:01.979 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:22:01.979 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:22:01.979 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730341049714742 collector_name = collector_ip = 10.0.0.31:8086 collector_list = [ 10.0.0.31:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/31/2024 02:22:01.979 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/31/2024 02:22:05.980 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/31/2024 02:22:05.982 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 10/31/2024 02:22:05.982 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/31/2024 02:22:05.982 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:22:05.982 7fdabfb4d848 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2183-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.31:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/31/2024 02:22:05.982 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/31/2024 02:22:05.983 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/31/2024 02:22:05.983 7fdabfb4d848 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 10/31/2024 02:22:05.983 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/31/2024 02:22:05.984 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/31/2024 02:22:05.984 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/31/2024 02:22:14.392 7fdabfa8d848 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 10/31/2024 02:22:14.393 7fdabfa8d848 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/31/2024 02:22:14.393 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 10/31/2024 02:22:14.394 7fdabfb4d848 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 10/31/2024 02:22:14.394 7fdabfb4d848 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/31/2024 02:22:14.394 7fdabfb4d848 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/31/2024 02:22:14.394 7fdabfb4d848 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/31/2024 02:22:14.394 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 10/31/2024 02:22:14.395 7fdabfb4d548 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/31/2024 02:22:14.397 7fdabfb4d548 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/31/2024 02:22:26.170 7fdabfa8d848 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 10/31/2024 02:22:26.171 7fdabfa8d848 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/31/2024 02:22:26.172 7fdabfb4d548 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 10/31/2024 02:22:26.172 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 10/31/2024 02:22:26.172 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/31/2024 02:22:26.172 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/31/2024 02:22:26.172 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/31/2024 02:22:26.173 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/31/2024 02:22:26.173 7fdabfb4d848 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/31/2024 02:22:30.348 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:22:32.694 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:22:38.439 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16487728, partition_space_available_1k=64602756, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2029084, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=160924, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54692, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8876724, free=6650412, buffers=712424, cached=16425560, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.35, five_min_avg=2.36, fifteen_min_avg=1.56, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:23:30.337 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:23:32.969 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:23:38.442 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17685556, partition_space_available_1k=63404928, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2014492, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168500, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46164, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8987188, free=20292648, buffers=737916, cached=2647368, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.32, five_min_avg=2.56, fifteen_min_avg=1.68, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:24:24.705 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/31/2024 02:24:26.806 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/31/2024 02:24:30.356 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:24:32.649 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:24:38.437 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18130044, partition_space_available_1k=62960440, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2019708, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=181304, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46216, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9550640, free=19187524, buffers=747012, cached=3179944, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=2.55, fifteen_min_avg=1.74, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:25:30.389 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:25:32.443 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:25:38.451 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19330716, partition_space_available_1k=61759768, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2032124, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185564, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46248, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10020008, free=17369012, buffers=779984, cached=4496116, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.01, five_min_avg=2.64, fifteen_min_avg=1.82, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:26:30.382 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:26:32.582 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:26:38.461 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19367512, partition_space_available_1k=61722972, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2035624, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188020, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46240, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10565820, free=16744168, buffers=780412, cached=4574720, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.45, five_min_avg=3.1, fifteen_min_avg=2.03, cpu_share=5.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:27:30.368 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:27:32.560 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:27:38.476 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19404976, partition_space_available_1k=61685508, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2036296, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=191004, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46260, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10340744, free=16940644, buffers=780452, cached=4603280, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.23, five_min_avg=3.28, fifteen_min_avg=2.16, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:28:30.363 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:28:32.494 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:28:38.472 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19493452, partition_space_available_1k=61597032, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2040768, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=192996, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46260, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11477448, free=15748936, buffers=780540, cached=4658196, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=3.32, fifteen_min_avg=2.25, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:29:30.397 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:29:32.552 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:29:38.483 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19491332, partition_space_available_1k=61599152, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2045892, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194176, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46268, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11117672, free=16081080, buffers=780772, cached=4685596, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.07, five_min_avg=3.23, fifteen_min_avg=2.29, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:30:30.380 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:30:32.526 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:30:38.487 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19512560, partition_space_available_1k=61577924, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2052312, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=196036, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46272, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10554908, free=16612804, buffers=780840, cached=4716568, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.08, five_min_avg=3.17, fifteen_min_avg=2.32, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:31:30.380 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:31:32.557 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:31:38.491 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19673604, partition_space_available_1k=61416880, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2048884, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197272, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46280, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10357892, free=16638292, buffers=780960, cached=4887976, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=3.27, fifteen_min_avg=2.41, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:32:30.386 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:32:32.562 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:32:38.506 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19696352, partition_space_available_1k=61394132, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2050640, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197840, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46284, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10369260, free=16609528, buffers=780968, cached=4905364, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=3.51, fifteen_min_avg=2.55, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:33:30.371 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/31/2024 02:33:32.526 7fdac62b9f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/31/2024 02:33:38.498 7fdac62b9f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2183-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19713992, partition_space_available_1k=61376492, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2053836, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=198220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46284, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10420748, free=16539248, buffers=780976, cached=4924148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.34, fifteen_min_avg=2.55, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/31/2024 02:34:11.121 7fdabfb4d548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map