01/20/2025 00:51:05.420 7efd93641df0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 01/20/2025 00:51:05.420 7efd93641df0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 01/20/2025 00:51:05.420 7efd93641df0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 01/20/2025 00:51:05.427 7efd8dcb3648 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 01/20/2025 00:51:05.428 7efd8dcb3648 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 01/20/2025 00:51:05.429 7efd8dcb3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 01/20/2025 00:51:05.429 7efd8dcb3e48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.430 7efd8dcb3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 01/20/2025 00:51:05.431 7efd8dcb3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.652 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 709459347163578444437678597872693606130860201679610839671625315281275153881, 'start': '1737334262000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 107572452164237107608285197302784037410039197687814003728389398901944899151952, 'start': '1737334262000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 37014433576836933640728098451325773561354718797200052825983083265433050321341, 'start': '1737334262000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 01/20/2025 00:51:05.809 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.809 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-16 01:14:15.868302", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7545-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 01/20/2025 00:51:05.809 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.810 7efd93641df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334265810190, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:51:05.810 7efd93641df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334265810487, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737334262000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737334262000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737334262000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-16 01:14:15.868302", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7545-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:51:05.813 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.813 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/20/2025 00:51:05.813 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.813 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:05.814 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737334262000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737334262000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737334262000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-16 01:14:15.868302", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7545-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 01/20/2025 00:51:05.814 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:06.259 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:51:07.024 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 00:51:07.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:07.027 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. database_usage = [ << disk_space_used_1k = 13719816 disk_space_available_1k = 67370668 analytics_db_size_1k = 4 >>, ] >> 01/20/2025 00:51:07.027 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:08.157 7efd93641df0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 01/20/2025 00:51:08.157 7efd93641df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334268157616, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:51:08.161 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:08.161 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/20/2025 00:51:08.162 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:09.430 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:09.432 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:09.432 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:09.432 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:09.432 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:09.432 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:09.432 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:09.433 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:13.433 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:13.434 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:13.434 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:13.434 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:13.435 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:13.436 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:13.436 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:13.436 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:13.577 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334273577754, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=1, pending=1, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:51:13.578 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 00:51:13.581 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:13.582 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 1 pending = 1 all_time_blocked = 0 >>, ] >> 01/20/2025 00:51:13.582 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:17.435 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:17.437 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:17.437 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:17.438 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:17.438 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:17.438 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:17.438 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:17.438 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:17.438 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:17.439 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:17.439 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:17.439 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:17.439 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:17.439 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:18.610 7efd93641df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334278610481, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737334262000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737334262000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737334262000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-16 01:14:15.868302", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7545-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:51:18.611 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13721396, partition_space_available_1k=67369088, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1244864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7492, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52404, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7445460, free=401940, buffers=656332, cached=7671904, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.88, five_min_avg=3.93, fifteen_min_avg=2.38, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:51:18.611 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:18.611 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737334262000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737334262000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737334262000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-16 01:14:15.868302", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7545-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 01/20/2025 00:51:18.612 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:18.612 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:18.612 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13721396 partition_space_available_1k = 67369088 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1244864 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7492 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 52404 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7445460 free = 401940 buffers = 656332 cached = 7671904 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.88 five_min_avg = 3.93 fifteen_min_avg = 2.38 cpu_share = 0 node_type = database-node >> >> 01/20/2025 00:51:18.612 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:21.447 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:21.468 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:21.468 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:21.468 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:21.469 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:21.469 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:21.469 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:21.469 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:21.470 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:21.470 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:21.470 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:21.471 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:21.471 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:21.471 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:25.471 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:25.473 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:25.473 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:25.473 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:25.473 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:25.474 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:25.474 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:25.474 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:25.474 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:25.474 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:25.474 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:25.475 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:25.475 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:25.475 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:29.475 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:29.477 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:29.477 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:29.477 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:29.478 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:29.479 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:29.479 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:29.479 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:33.479 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:33.481 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:33.481 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:33.481 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:33.482 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:33.483 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:33.483 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:33.483 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:37.482 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:37.483 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:37.484 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:37.485 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:37.485 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:37.485 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:37.485 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:37.485 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:41.487 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:41.489 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:41.489 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:41.489 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:41.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:41.491 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:41.491 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:41.491 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:45.492 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:45.493 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:45.494 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:45.494 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:45.495 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:45.495 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:45.495 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:45.496 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:45.496 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:45.497 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:45.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:45.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:45.498 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:45.498 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:49.495 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:49.497 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:49.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:49.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:49.497 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:49.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:49.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:49.498 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:53.497 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:53.499 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:53.499 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:53.499 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:53.500 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:53.501 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:53.501 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:53.501 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:57.499 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:51:57.501 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 00:51:57.501 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:51:57.502 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:57.502 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:51:57.502 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:51:57.502 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:51:57.503 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:01.507 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:52:01.508 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 00:52:01.509 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:52:01.509 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:52:01.509 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:52:01.509 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:52:01.509 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 00:52:01.510 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 00:52:01.510 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 00:52:01.510 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:01.511 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:06.553 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:52:07.994 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:09.431 7efd93641df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334329431432, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.42:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:52:09.441 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:14.502 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334334502656, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=1, pending=1, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=1, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=1, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:52:14.509 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:16.580 7efd8de00348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 01/20/2025 00:52:16.580 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 01/20/2025 00:52:16.581 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 01/20/2025 00:52:16.581 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:16.581 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 01/20/2025 00:52:16.581 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:16.582 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:16.582 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 01/20/2025 00:52:16.582 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:19.539 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14008276, partition_space_available_1k=67082208, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1318632, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9428, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52432, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7672508, free=455284, buffers=662408, cached=7385436, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.33, five_min_avg=4.41, fifteen_min_avg=2.64, cpu_share=19.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:52:19.540 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:19.540 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14008276 partition_space_available_1k = 67082208 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1318632 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 9428 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 52432 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7672508 free = 455284 buffers = 662408 cached = 7385436 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.33 five_min_avg = 4.41 fifteen_min_avg = 2.64 cpu_share = 19.0 node_type = database-node >> >> 01/20/2025 00:52:19.540 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 00:52:20.581 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 00:52:20.583 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 00:52:20.583 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 00:52:20.584 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:52:20.584 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 00:52:20.585 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 00:52:20.585 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 00:52:20.585 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 00:52:20.586 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 00:52:20.586 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:20.586 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 00:52:20.592 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 01/20/2025 00:52:20.593 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 00:52:20.594 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 01/20/2025 00:52:20.594 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 01/20/2025 00:52:20.594 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 00:52:20.595 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 00:52:20.595 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 00:52:20.598 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 00:52:20.599 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 01/20/2025 00:52:20.600 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 00:52:20.602 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 00:52:28.488 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 01/20/2025 00:52:28.488 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 00:52:28.489 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 01/20/2025 00:52:28.490 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 01/20/2025 00:52:28.490 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 00:52:28.490 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 00:52:28.490 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 00:52:28.491 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 00:52:28.491 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 00:52:28.494 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 00:53:06.215 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:53:11.320 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737334391320887, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 00:53:16.361 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14160016, partition_space_available_1k=66930468, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10148, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8668688, free=349408, buffers=670924, cached=6486616, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.7, five_min_avg=5.56, fifteen_min_avg=3.15, cpu_share=21.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:54:06.293 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:54:17.204 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14218904, partition_space_available_1k=66871580, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1328144, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10280, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8519736, free=455096, buffers=671232, cached=6529572, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.3, five_min_avg=5.43, fifteen_min_avg=3.26, cpu_share=16.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:55:06.250 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:55:16.228 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14244576, partition_space_available_1k=66845908, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363776, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10364, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52088, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8579492, free=374468, buffers=671676, cached=6550000, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.34, five_min_avg=5.32, fifteen_min_avg=3.36, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:56:06.490 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:56:10.351 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 00:56:17.596 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14706868, partition_space_available_1k=66383616, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339424, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10516, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52124, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8643592, free=403476, buffers=676424, cached=6452144, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.66, five_min_avg=5.15, fifteen_min_avg=3.42, cpu_share=16.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:56:47.647 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 00:57:06.244 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:57:17.109 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14715420, partition_space_available_1k=66375064, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1345660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10624, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52152, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8639640, free=419676, buffers=676456, cached=6439864, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.61, five_min_avg=4.97, fifteen_min_avg=3.47, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:58:06.208 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:58:15.879 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14719064, partition_space_available_1k=66371420, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1371540, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10748, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52164, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8713416, free=349196, buffers=676660, cached=6436364, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.96, five_min_avg=4.68, fifteen_min_avg=3.46, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 00:59:06.241 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 00:59:15.968 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14733700, partition_space_available_1k=66356784, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358024, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10544, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8743304, free=349256, buffers=675968, cached=6407108, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.13, five_min_avg=4.84, fifteen_min_avg=3.59, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:00:06.260 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:00:17.148 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14745732, partition_space_available_1k=66344752, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1361408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10564, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52188, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8793964, free=380220, buffers=675956, cached=6325496, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.1, five_min_avg=4.55, fifteen_min_avg=3.56, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:01:06.256 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:01:16.071 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14753204, partition_space_available_1k=66337280, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1368312, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10640, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52188, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8851840, free=331200, buffers=676036, cached=6316560, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.31, five_min_avg=4.75, fifteen_min_avg=3.69, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:02:06.296 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:02:16.553 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14764048, partition_space_available_1k=66326436, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1375776, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10708, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8889836, free=369264, buffers=676132, cached=6240404, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.19, five_min_avg=4.79, fifteen_min_avg=3.77, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:03:06.262 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:03:17.054 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14768156, partition_space_available_1k=66322328, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1380016, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10776, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52188, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8909744, free=404216, buffers=676080, cached=6185596, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.66, five_min_avg=4.72, fifteen_min_avg=3.81, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:04:06.222 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:04:16.233 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14778180, partition_space_available_1k=66312304, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1383384, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10792, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52204, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8968124, free=342272, buffers=676076, cached=6189164, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.58, five_min_avg=4.67, fifteen_min_avg=3.85, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:05:06.311 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:05:16.094 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14788604, partition_space_available_1k=66301880, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1388784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52560, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8987320, free=347772, buffers=675852, cached=6164692, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.96, five_min_avg=5.05, fifteen_min_avg=4.03, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:06:06.351 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:06:17.345 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14800452, partition_space_available_1k=66290032, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1402788, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10748, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52564, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8909836, free=420972, buffers=675904, cached=6168924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.42, five_min_avg=4.77, fifteen_min_avg=4.0, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:07:06.270 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:07:16.109 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14809284, partition_space_available_1k=66281200, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1399568, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10796, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52572, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8929256, free=391412, buffers=675924, cached=6179044, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.97, five_min_avg=4.56, fifteen_min_avg=3.97, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:08:06.297 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:08:16.163 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14814848, partition_space_available_1k=66275636, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1404872, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10820, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52564, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8910080, free=400828, buffers=675928, cached=6188800, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=4.36, fifteen_min_avg=3.94, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:09:06.339 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:09:17.735 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14825944, partition_space_available_1k=66264540, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1409616, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10876, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52572, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8939032, free=407780, buffers=675856, cached=6152968, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.7, five_min_avg=4.22, fifteen_min_avg=3.92, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:10:06.279 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:10:16.409 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14834860, partition_space_available_1k=66255624, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1413008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8928120, free=414676, buffers=675856, cached=6156984, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.82, five_min_avg=4.14, fifteen_min_avg=3.91, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:11:06.337 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:11:16.206 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14840248, partition_space_available_1k=66250236, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1423652, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10780, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8955508, free=377864, buffers=675864, cached=6166400, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.38, five_min_avg=4.21, fifteen_min_avg=3.95, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:12:06.285 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:12:17.824 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14829500, partition_space_available_1k=66260984, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1427028, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10788, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8967928, free=407956, buffers=675732, cached=6124020, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.72, five_min_avg=4.54, fifteen_min_avg=4.08, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:13:06.303 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:13:16.152 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14837192, partition_space_available_1k=66253292, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1433668, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10860, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8977076, free=390984, buffers=675748, cached=6131828, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.65, five_min_avg=4.42, fifteen_min_avg=4.06, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:14:06.328 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:14:16.052 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14841396, partition_space_available_1k=66249088, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1435076, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10864, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8989144, free=369916, buffers=675748, cached=6140828, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.21, five_min_avg=4.3, fifteen_min_avg=4.04, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:15:06.407 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:15:18.450 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14854340, partition_space_available_1k=66236144, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1442268, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10864, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52576, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8970064, free=410012, buffers=675700, cached=6119860, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=4.23, fifteen_min_avg=4.03, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:16:06.285 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:16:16.094 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14862388, partition_space_available_1k=66228096, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1449864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10876, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8997344, free=375440, buffers=675700, cached=6127152, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.96, five_min_avg=4.14, fifteen_min_avg=4.01, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:17:06.374 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:17:15.986 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14867628, partition_space_available_1k=66222856, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1458392, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10776, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8991712, free=375360, buffers=675696, cached=6132868, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.76, five_min_avg=4.05, fifteen_min_avg=3.99, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:18:06.379 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:18:18.089 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14871124, partition_space_available_1k=66219360, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1457208, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10820, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8991240, free=397604, buffers=675648, cached=6111144, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.97, five_min_avg=4.04, fifteen_min_avg=3.99, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:19:06.307 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:19:16.064 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14883880, partition_space_available_1k=66206604, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1462796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10840, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9007696, free=378364, buffers=675660, cached=6113916, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=3.92, fifteen_min_avg=3.95, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:20:06.305 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:20:16.223 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14889572, partition_space_available_1k=66200912, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1466304, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10864, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52624, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9008676, free=371456, buffers=675664, cached=6119840, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=3.73, fifteen_min_avg=3.89, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:21:06.402 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:21:16.206 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14896180, partition_space_available_1k=66194304, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1471564, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10872, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52624, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8999992, free=380472, buffers=675656, cached=6119516, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.11, five_min_avg=3.95, fifteen_min_avg=3.96, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:22:06.336 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:22:15.997 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14902904, partition_space_available_1k=66187580, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1483996, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10908, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9025888, free=341140, buffers=675656, cached=6132952, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=4.01, fifteen_min_avg=3.98, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:23:06.345 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:23:16.028 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14914756, partition_space_available_1k=66175728, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1482712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10852, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9010712, free=360256, buffers=675640, cached=6129028, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=3.93, fifteen_min_avg=3.96, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:24:06.389 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:24:16.404 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14915992, partition_space_available_1k=66174492, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1490864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52560, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9042624, free=387352, buffers=675524, cached=6070136, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=3.66, fifteen_min_avg=3.86, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:25:06.365 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:25:17.002 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14917904, partition_space_available_1k=66172580, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1490904, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52568, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9022312, free=409644, buffers=675488, cached=6068192, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=3.59, fifteen_min_avg=3.82, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:26:06.467 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:26:16.145 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14927832, partition_space_available_1k=66162652, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1496056, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52568, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9046224, free=380400, buffers=675488, cached=6073524, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.41, five_min_avg=3.57, fifteen_min_avg=3.8, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:27:06.395 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:27:15.942 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14932956, partition_space_available_1k=66157528, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1501356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52572, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9029356, free=385828, buffers=675492, cached=6084960, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.89, five_min_avg=3.66, fifteen_min_avg=3.82, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:28:06.412 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:28:17.289 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14940700, partition_space_available_1k=66149784, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503400, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10940, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9036728, free=403620, buffers=675420, cached=6059868, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.56, five_min_avg=4.11, fifteen_min_avg=3.96, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:29:06.383 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:29:16.315 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14945168, partition_space_available_1k=66145316, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1509728, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10876, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52576, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9034204, free=396436, buffers=675428, cached=6069568, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.28, five_min_avg=3.99, fifteen_min_avg=3.93, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:30:06.449 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:30:16.551 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14955532, partition_space_available_1k=66134952, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1515832, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10960, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52596, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9070124, free=355628, buffers=675432, cached=6074452, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.11, five_min_avg=4.0, fifteen_min_avg=3.93, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:31:06.484 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:31:17.034 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14974272, partition_space_available_1k=66116212, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1498628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10984, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9017320, free=387244, buffers=675432, cached=6095640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.26, five_min_avg=4.07, fifteen_min_avg=3.96, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:32:06.392 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:32:16.063 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14983988, partition_space_available_1k=66106496, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1499916, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10988, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9036440, free=364136, buffers=675436, cached=6099624, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.94, five_min_avg=4.33, fifteen_min_avg=4.06, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:33:06.423 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:33:16.031 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14985532, partition_space_available_1k=66104952, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1505320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10992, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52592, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9018688, free=381328, buffers=675436, cached=6100184, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.07, five_min_avg=4.51, fifteen_min_avg=4.14, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:34:06.409 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:34:16.262 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14996068, partition_space_available_1k=66094416, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1508204, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11008, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9018744, free=376228, buffers=675436, cached=6105228, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.14, five_min_avg=4.34, fifteen_min_avg=4.1, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:35:06.542 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:35:16.697 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15002652, partition_space_available_1k=66087832, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1512720, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10916, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52592, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9006844, free=413992, buffers=675388, cached=6079412, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.04, five_min_avg=4.28, fifteen_min_avg=4.1, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:36:06.448 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:36:16.101 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15013208, partition_space_available_1k=66077276, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1517988, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10924, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9038016, free=381028, buffers=675388, cached=6081204, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=4.16, fifteen_min_avg=4.07, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:37:06.442 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:37:16.045 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15013384, partition_space_available_1k=66077100, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1521828, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10944, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9028984, free=395196, buffers=675368, cached=6076088, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.51, five_min_avg=4.02, fifteen_min_avg=4.03, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:38:06.445 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:38:17.559 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15021748, partition_space_available_1k=66068736, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1526656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52572, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9032992, free=399568, buffers=675304, cached=6067772, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.82, five_min_avg=3.99, fifteen_min_avg=4.01, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:39:06.480 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:39:16.226 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15029788, partition_space_available_1k=66060696, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1531772, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10976, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9058736, free=365356, buffers=675308, cached=6076236, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.74, five_min_avg=3.95, fifteen_min_avg=3.99, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:40:06.457 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:40:16.331 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15040136, partition_space_available_1k=66050348, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1536576, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10980, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9054288, free=369376, buffers=675308, cached=6076664, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.41, five_min_avg=3.84, fifteen_min_avg=3.96, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:41:06.600 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:41:17.439 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15041524, partition_space_available_1k=66048960, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1541336, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10896, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52576, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9043288, free=399316, buffers=675292, cached=6057740, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.73, five_min_avg=3.86, fifteen_min_avg=3.95, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:42:06.499 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:42:16.111 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15051084, partition_space_available_1k=66039400, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1548228, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10936, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9049412, free=387368, buffers=675292, cached=6063564, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.21, five_min_avg=3.96, fifteen_min_avg=3.98, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:43:06.510 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:43:16.075 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15056876, partition_space_available_1k=66033608, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1553596, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10944, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9065632, free=361468, buffers=675292, cached=6073244, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.62, five_min_avg=4.12, fifteen_min_avg=4.04, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:44:06.622 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:44:16.485 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15067992, partition_space_available_1k=66022492, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1559412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10960, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52592, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9048108, free=372940, buffers=675296, cached=6079292, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.65, five_min_avg=4.28, fifteen_min_avg=4.1, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:45:06.539 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:45:16.382 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15073528, partition_space_available_1k=66016956, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1561320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11004, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9067884, free=361176, buffers=675288, cached=6071288, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.06, five_min_avg=4.16, fifteen_min_avg=4.07, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:46:06.628 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:46:18.169 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15409556, partition_space_available_1k=65680928, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1549856, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11036, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52440, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9201808, free=385172, buffers=685604, cached=5903052, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.35, five_min_avg=4.52, fifteen_min_avg=4.2, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:47:03.733 7efd8de00348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 01/20/2025 01:47:03.733 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 01/20/2025 01:47:03.737 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 01:47:03.737 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 01/20/2025 01:47:03.737 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:47:03.738 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 01/20/2025 01:47:03.739 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:47:03.739 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 01:47:03.739 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 01:47:03.742 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 01:47:03.746 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:47:03.746 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:47:04.085 7efd8de00348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 01/20/2025 01:47:04.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 01/20/2025 01:47:04.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 01/20/2025 01:47:04.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:47:04.086 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 01/20/2025 01:47:04.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:47:04.087 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:47:04.087 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:47:04.087 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:47:06.622 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:47:07.171 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:47:07.172 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. database_usage = [ << disk_space_used_1k = 15436256 disk_space_available_1k = 65654228 analytics_db_size_1k = 14508 >>, ] >> 01/20/2025 01:47:07.172 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:47:08.091 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:47:08.126 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 01:47:08.128 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:47:08.128 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:47:08.129 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:47:08.129 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:47:08.129 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 01:47:08.129 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 01:47:08.129 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 01:47:08.130 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:47:08.130 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:47:08.147 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 01/20/2025 01:47:08.147 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 01:47:08.148 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 01/20/2025 01:47:08.149 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 01/20/2025 01:47:08.149 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 01:47:08.150 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 01:47:08.150 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 01:47:08.150 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 01:47:08.151 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 01/20/2025 01:47:08.151 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 01:47:08.153 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 01:47:15.387 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15436604, partition_space_available_1k=65653880, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1552552, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11728, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52424, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9314136, free=350348, buffers=689140, cached=5822012, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.06, five_min_avg=4.62, fifteen_min_avg=4.26, cpu_share=17.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:48:06.687 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:48:22.700 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737337702700122, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=1, pending=1, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=2, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:48:27.785 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16420112, partition_space_available_1k=64670372, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1573768, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12236, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9394868, free=386624, buffers=695344, cached=5698800, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.1, five_min_avg=6.07, fifteen_min_avg=4.77, cpu_share=21.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:49:06.650 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:49:15.429 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737337755429039, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:49:20.463 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17152108, partition_space_available_1k=63938376, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1475244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12236, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51704, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9332068, free=364048, buffers=747768, cached=5731752, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.34, five_min_avg=6.31, fifteen_min_avg=4.94, cpu_share=20.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:50:06.623 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:50:15.019 7efd8de00348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 01/20/2025 01:50:15.019 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 01/20/2025 01:50:15.024 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 01:50:15.024 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 01/20/2025 01:50:15.024 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:15.025 7efd8cf4a048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 01/20/2025 01:50:15.025 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:15.025 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 01:50:15.025 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 01:50:15.026 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 01:50:15.026 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:50:15.026 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:50:15.139 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 01/20/2025 01:50:15.140 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 01:50:15.140 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 01/20/2025 01:50:15.141 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 01/20/2025 01:50:15.141 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 01:50:15.141 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 01:50:15.141 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 01:50:15.141 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 01:50:15.142 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 01/20/2025 01:50:15.142 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 01:50:15.157 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 01:50:15.235 7efd8de00348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 01/20/2025 01:50:15.235 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 01/20/2025 01:50:15.243 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:15.243 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 01/20/2025 01:50:15.243 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:15.243 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 01/20/2025 01:50:15.243 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:15.244 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:15.244 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:15.244 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:15.244 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:15.245 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:15.245 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:15.245 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:15.245 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:18.618 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17873588, partition_space_available_1k=63216896, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1464136, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12564, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51528, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8669128, free=1002636, buffers=716544, cached=5787328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.32, five_min_avg=6.53, fifteen_min_avg=5.11, cpu_share=20.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:50:18.618 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:18.619 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17873588 partition_space_available_1k = 63216896 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1464136 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12564 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51528 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 8669128 free = 1002636 buffers = 716544 cached = 5787328 node_type = database-node >> system_cpu_usage = << one_min_avg = 7.32 five_min_avg = 6.53 fifteen_min_avg = 5.11 cpu_share = 20.0 node_type = database-node >> >> 01/20/2025 01:50:18.619 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:19.244 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:19.252 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 01:50:19.252 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:19.252 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:50:19.253 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:50:27.999 7efd8de00348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 01/20/2025 01:50:28.000 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 01/20/2025 01:50:28.000 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 01/20/2025 01:50:28.000 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:28.000 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 01/20/2025 01:50:28.001 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:28.001 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:28.001 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:28.001 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:32.004 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:32.006 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:32.006 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:32.006 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:32.006 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:32.006 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:32.006 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:32.007 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:36.010 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:36.014 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:36.015 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:36.015 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:36.015 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:36.015 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:36.015 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:36.016 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:36.016 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:36.016 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:36.016 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:36.016 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:36.016 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:36.017 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:40.015 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:40.017 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:40.017 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:40.017 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:40.017 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:40.017 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:40.017 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:40.018 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:44.020 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:44.021 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:44.021 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:44.021 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:44.021 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:44.021 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:44.021 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:44.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:48.024 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:48.025 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:48.026 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:48.027 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:48.027 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:48.027 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:48.027 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:52.027 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:52.029 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:52.029 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:52.029 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:52.029 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:52.029 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:52.029 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:52.030 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:56.030 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:50:56.033 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:50:56.033 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:50:56.034 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:56.035 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:50:56.035 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:50:56.035 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:00.036 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:00.038 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:00.038 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:00.038 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:00.038 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:00.038 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:00.038 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:00.039 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:04.042 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:04.043 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:04.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:04.045 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:04.045 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:04.045 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:04.045 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:04.045 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:06.477 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:51:06.873 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:06.874 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. database_usage = [ << disk_space_used_1k = 17770088 disk_space_available_1k = 63320396 analytics_db_size_1k = 25288 >>, ] >> 01/20/2025 01:51:06.874 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:08.044 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:08.046 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:08.046 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:08.046 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:08.046 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:08.046 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:08.046 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:08.047 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:08.047 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:08.047 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:08.047 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:08.047 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:08.047 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:08.048 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:12.047 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:12.049 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:12.050 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:12.050 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:12.050 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:12.050 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:12.050 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:12.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:12.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:12.051 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:12.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:12.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:12.052 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:12.052 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:15.489 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17770436, partition_space_available_1k=63320048, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1465776, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12924, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51604, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7451156, free=2266360, buffers=716928, cached=5741192, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=5.76, fifteen_min_avg=4.92, cpu_share=9.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:51:15.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:15.490 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17770436 partition_space_available_1k = 63320048 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1465776 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12924 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51604 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7451156 free = 2266360 buffers = 716928 cached = 5741192 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.08 five_min_avg = 5.76 fifteen_min_avg = 4.92 cpu_share = 9.75 node_type = database-node >> >> 01/20/2025 01:51:15.490 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:16.050 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:16.051 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:16.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:16.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:16.051 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:16.051 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:16.052 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:16.053 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:20.053 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:20.055 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:20.055 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:20.056 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:20.056 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:20.056 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:20.056 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:20.057 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:20.059 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:20.059 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:20.060 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:20.060 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:20.060 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:20.060 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:24.057 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:24.058 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:24.058 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:24.058 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:24.058 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:24.059 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:24.059 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:24.059 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:24.059 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:24.059 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:24.060 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:24.060 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:24.060 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:24.060 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:28.059 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:28.061 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:28.061 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:28.062 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:28.062 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:28.062 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:28.062 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:28.062 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:28.062 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:28.063 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:28.063 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:28.063 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:28.063 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:28.063 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:32.065 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:32.068 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:32.068 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:32.068 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:32.069 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:32.069 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:32.069 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:32.069 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:32.069 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:32.070 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:32.070 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:32.070 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:32.070 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:32.070 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:36.070 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:36.074 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:36.074 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:36.074 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:36.074 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:36.074 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:36.074 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:36.075 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:36.075 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:36.075 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:36.075 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:36.075 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:36.075 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:36.076 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:40.080 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:40.081 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:40.081 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:40.081 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:40.081 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:40.081 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:40.081 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:40.082 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:40.082 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:40.082 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:40.082 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:40.082 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:40.083 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:40.083 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:44.083 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:44.085 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:44.085 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:44.085 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:44.085 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:44.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:44.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:44.086 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:44.087 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:44.087 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:44.087 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:44.087 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:44.087 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:44.088 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:48.087 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:48.088 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:48.089 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:48.089 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:48.089 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:48.089 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:48.089 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:48.090 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:52.091 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:52.092 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:52.093 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:52.094 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:52.094 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:52.094 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:52.094 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:52.094 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:56.112 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:51:56.176 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:51:56.176 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:51:56.176 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:56.176 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:51:56.177 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:51:56.177 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:51:56.178 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:51:56.178 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:56.178 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:51:56.178 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:56.179 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:51:56.179 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:51:56.179 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:00.182 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:00.192 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:52:00.192 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:00.192 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:00.192 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:00.192 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:00.192 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:52:00.193 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:04.193 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:04.194 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:52:04.194 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:04.195 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:04.195 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:04.195 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:04.195 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:52:04.195 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:52:04.196 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:04.196 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:52:04.196 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:04.196 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:04.196 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:52:04.197 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:06.504 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:52:07.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:07.022 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. database_usage = [ << disk_space_used_1k = 17806104 disk_space_available_1k = 63284380 analytics_db_size_1k = 25584 >>, ] >> 01/20/2025 01:52:07.022 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:08.195 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:08.196 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:52:08.197 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:08.197 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:08.197 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:08.197 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:08.197 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:52:08.198 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:52:08.198 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:08.198 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:52:08.198 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:08.198 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:08.199 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:52:08.199 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:12.197 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:12.200 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:52:12.200 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:12.200 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:12.201 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:12.201 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:12.201 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:52:12.201 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:52:12.202 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:12.202 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:52:12.202 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:12.202 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:12.203 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:52:12.203 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:16.201 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:16.202 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:52:16.202 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:16.202 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:16.203 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:16.203 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:16.203 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:52:16.204 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:52:16.204 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:16.204 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:52:16.204 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:16.204 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:16.205 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:52:16.205 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:17.106 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17807004, partition_space_available_1k=63283480, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1462208, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13188, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51700, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8079164, free=1572908, buffers=720500, cached=5803064, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.28, five_min_avg=6.04, fifteen_min_avg=5.07, cpu_share=19.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:52:17.107 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:17.107 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17807004 partition_space_available_1k = 63283480 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1462208 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 13188 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51700 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 8079164 free = 1572908 buffers = 720500 cached = 5803064 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.28 five_min_avg = 6.04 fifteen_min_avg = 5.07 cpu_share = 19.0 node_type = database-node >> >> 01/20/2025 01:52:17.109 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:20.204 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:20.206 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 01/20/2025 01:52:20.206 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:20.206 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:20.206 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:20.206 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:20.207 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 01/20/2025 01:52:20.207 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 01/20/2025 01:52:20.207 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:20.207 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 01/20/2025 01:52:20.208 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:20.208 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:20.208 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1737334265430301 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.49:8086, 10.0.0.42:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 01/20/2025 01:52:20.208 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:52:24.208 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:52:24.216 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 01:52:24.216 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:52:24.216 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:24.216 7efd8cf4a348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:52:24.216 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:52:24.216 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 01:52:24.217 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 01:52:24.217 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 01:52:24.217 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:52:24.217 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:52:27.152 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 01/20/2025 01:52:27.153 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 01:52:27.153 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 01/20/2025 01:52:27.154 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 01/20/2025 01:52:27.154 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 01:52:27.154 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 01:52:27.154 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 01:52:27.154 7efd8cf4a348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 01:52:27.154 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 01/20/2025 01:52:27.155 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 01:52:27.160 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 01:52:37.129 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001584 01/20/2025 01:52:37.131 7efd8de00348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 01:52:37.136 7efd8cf4a048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 01/20/2025 01:52:37.137 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 01/20/2025 01:52:37.137 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 01:52:37.138 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 01:52:37.138 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 01:52:37.143 7efd8cf4a048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 01:52:37.144 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 01:52:37.144 7efd8cf4a348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 01:53:06.534 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:53:15.508 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18343424, partition_space_available_1k=62747060, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1460068, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12344, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51632, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8960516, free=522052, buffers=734404, cached=5958664, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.05, five_min_avg=6.17, fifteen_min_avg=5.18, cpu_share=18.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:54:06.541 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:54:15.575 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18366516, partition_space_available_1k=62723968, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1467884, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12368, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51628, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9051036, free=418384, buffers=735808, cached=5970408, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.23, five_min_avg=5.58, fifteen_min_avg=5.04, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:55:06.530 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:55:16.099 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18372204, partition_space_available_1k=62718280, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1472408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12436, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51516, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9066392, free=296500, buffers=726544, cached=6086200, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.43, five_min_avg=5.1, fifteen_min_avg=4.91, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:56:06.731 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:56:18.590 7efd93641df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19427424, partition_space_available_1k=61663060, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1465120, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12480, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51472, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7930348, free=1069028, buffers=712784, cached=6463476, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.57, five_min_avg=5.26, fifteen_min_avg=4.97, cpu_share=17.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:56:53.354 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 01/20/2025 01:56:53.354 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 01/20/2025 01:56:53.354 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 01/20/2025 01:56:53.364 7f0bf96eaa20 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 01/20/2025 01:56:53.364 7f0bf96eaa20 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 01/20/2025 01:56:53.365 7f0bf96ed390 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 01/20/2025 01:56:53.366 7f0bf96ed390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 01:56:53.366 7f0bf96ed390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 01:56:53.366 7f0bf96ed390 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 01/20/2025 01:56:53.366 7f0bf96ed390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.366 7f0bf96ed390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/20/2025 01:56:53.367 7f0bf96ed390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.367 7f0bf96ed390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.367 7f0bf96ed390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1737338213366476 collector_name = collector_ip = collector_list = [ 10.0.0.22:8086, 10.0.0.49:8086, 10.0.0.42:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 01/20/2025 01:56:53.367 7f0bf96ed390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.631 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 47503145808947257505166551486271090021270647674855529858735948382071697057921, 'start': '1737338210000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 92742334923284872218126564997346643462638360677740146817329425189002186403345, 'start': '1737338210000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 22605918936253463537121059498655882359288515417226215242615586951118411259845, 'start': '1737338210000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 01/20/2025 01:56:53.824 7f0c02000a20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.827 7f0c02000a20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-01-20 00:44:39.080355", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7550-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 01/20/2025 01:56:53.827 7f0c02000a20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.828 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338213827914, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:56:53.828 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338213828301, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737338210000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737338210000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737338210000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-01-20 00:44:39.080355", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7550-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:56:53.830 7f0bf96d0198 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.830 7f0bf96d0198 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/20/2025 01:56:53.830 7f0bf96d0198 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.830 7f0bf96d0198 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:53.830 7f0bf96d0198 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737338210000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737338210000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1737338210000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-01-20 00:44:39.080355", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7550-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 01/20/2025 01:56:53.830 7f0bf96d0198 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:54.588 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:56:55.234 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 01:56:55.236 7f0bf979b588 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:55.236 7f0bf979b588 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. database_usage = [ << disk_space_used_1k = 19412712 disk_space_available_1k = 61677772 analytics_db_size_1k = 41232 >>, ] >> 01/20/2025 01:56:55.236 7f0bf979b588 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:55.885 7f0bfbde5808 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 01/20/2025 01:56:55.885 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338215885487, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:56:55.887 7f0bf3975eb8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:55.888 7f0bf3975eb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 01/20/2025 01:56:55.888 7f0bf3975eb8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 01/20/2025 01:56:57.367 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 01/20/2025 01:56:57.369 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 01/20/2025 01:56:57.369 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 01/20/2025 01:56:57.369 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:56:57.369 7f0bf39a1630 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2650-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 01/20/2025 01:56:57.369 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 01/20/2025 01:56:57.369 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 01/20/2025 01:56:57.370 7f0bf39a1630 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 01/20/2025 01:56:57.370 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 01/20/2025 01:56:57.371 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:56:57.371 7f0bf39a1630 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 01/20/2025 01:56:57.377 7f0bf39a1588 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 01/20/2025 01:56:57.378 7f0bf39a1588 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 01/20/2025 01:56:57.378 7f0bf96d08d0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 01/20/2025 01:56:57.378 7f0bf96d08d0 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 01/20/2025 01:56:57.379 7f0bf96d08d0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 01/20/2025 01:56:57.379 7f0bf96d08d0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 01:56:57.379 7f0bf96d08d0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 01/20/2025 01:56:57.379 7f0bf96d08d0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 01/20/2025 01:56:57.380 7f0bf96d0f60 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 01/20/2025 01:56:57.381 7f0bf96d0f60 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 01/20/2025 01:57:01.444 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338221444780, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=1), thread_pool_stats=[CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=1, pending=1, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:57:01.446 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 01/20/2025 01:57:06.476 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338226476475, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737338210000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737338210000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1737338210000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-01-20 00:44:39.080355", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7550-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:57:06.477 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19415704, partition_space_available_1k=61674780, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1279076, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9760, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51440, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8431996, free=568992, buffers=715004, cached=6459644, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.15, five_min_avg=5.46, fifteen_min_avg=5.05, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:57:54.211 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:57:55.602 7f0bfbde5808 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338275602054, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.22:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:57:58.399 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2650-1.', _node_type='Database', _instance_id='0', _timestamp=1737338278399144, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 01/20/2025 01:58:03.428 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19419612, partition_space_available_1k=61670872, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1317156, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10012, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51468, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8472628, free=521984, buffers=715136, cached=6465888, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.4, five_min_avg=5.42, fifteen_min_avg=5.06, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:58:54.622 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 01:59:04.374 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19422568, partition_space_available_1k=61667916, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1328792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10272, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51724, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8530240, free=461128, buffers=715144, cached=6469124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=4.84, fifteen_min_avg=4.88, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 01:59:54.350 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:00:04.010 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19427552, partition_space_available_1k=61662932, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333124, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11220, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51876, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8511032, free=474940, buffers=715156, cached=6474508, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.74, five_min_avg=4.36, fifteen_min_avg=4.71, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:00:54.278 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:01:04.246 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19474416, partition_space_available_1k=61616068, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341580, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11436, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51864, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8659336, free=315364, buffers=713960, cached=6486976, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.74, five_min_avg=4.32, fifteen_min_avg=4.67, cpu_share=17.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:01:54.309 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:02:04.213 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19483272, partition_space_available_1k=61607212, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1381244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11444, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51796, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8667256, free=383356, buffers=709884, cached=6415140, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.07, five_min_avg=4.33, fifteen_min_avg=4.66, cpu_share=19.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:02:54.265 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:03:04.119 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19488496, partition_space_available_1k=61601988, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356844, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10780, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51732, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8408484, free=695912, buffers=714944, cached=6356296, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.89, five_min_avg=4.21, fifteen_min_avg=4.59, cpu_share=16.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:03:54.325 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:04:05.492 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19484620, partition_space_available_1k=61605864, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10860, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51748, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8619476, free=471660, buffers=715988, cached=6368512, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=4.01, fifteen_min_avg=4.5, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:04:54.341 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:05:04.606 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19490140, partition_space_available_1k=61600344, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1388164, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11000, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51752, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8600756, free=478860, buffers=717784, cached=6378236, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.64, five_min_avg=4.19, fifteen_min_avg=4.52, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:05:54.368 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:06:04.088 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19496452, partition_space_available_1k=61594032, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1375160, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11076, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51692, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8672216, free=460916, buffers=713824, cached=6328680, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.89, five_min_avg=4.05, fifteen_min_avg=4.46, cpu_share=16.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:06:54.293 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:07:06.295 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19502972, partition_space_available_1k=61587512, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1381636, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11088, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51648, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8674956, free=493528, buffers=711980, cached=6295172, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.99, five_min_avg=3.99, fifteen_min_avg=4.4, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:07:54.735 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:08:06.186 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19509712, partition_space_available_1k=61580772, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1398644, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11104, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51652, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8724500, free=428152, buffers=712804, cached=6310180, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.97, five_min_avg=3.94, fifteen_min_avg=4.36, cpu_share=17.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:08:54.283 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:09:04.117 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19530556, partition_space_available_1k=61559928, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1392656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11056, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51736, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8757180, free=428004, buffers=710948, cached=6279504, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.29, five_min_avg=4.07, fifteen_min_avg=4.38, cpu_share=18.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:09:54.461 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:10:05.085 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19547648, partition_space_available_1k=61542836, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1399024, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11148, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51848, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8822016, free=319076, buffers=713364, cached=6321180, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.75, five_min_avg=4.22, fifteen_min_avg=4.41, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:10:54.389 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:11:04.090 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19548432, partition_space_available_1k=61542052, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1405040, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11192, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51860, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8739236, free=399712, buffers=713428, cached=6323260, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=3.93, fifteen_min_avg=4.3, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:11:54.355 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:12:05.045 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19552816, partition_space_available_1k=61537668, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1409840, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11216, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51824, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8692716, free=473812, buffers=711332, cached=6297776, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=3.99, fifteen_min_avg=4.29, cpu_share=16.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:12:54.354 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:13:03.648 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19557740, partition_space_available_1k=61532744, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1419972, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11256, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51828, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8760660, free=400744, buffers=711348, cached=6302884, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=3.79, fifteen_min_avg=4.2, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:13:54.332 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:13:59.406 7f0bf966dc18 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:14:04.071 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19918976, partition_space_available_1k=61171508, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1424512, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11344, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51640, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8631156, free=456108, buffers=706732, cached=6381640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.42, five_min_avg=4.06, fifteen_min_avg=4.27, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:14:27.295 7f0bfbccf978 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:14:45.577 7f0c02900438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:14:54.323 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:15:03.251 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19919976, partition_space_available_1k=61170508, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1427788, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11268, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51644, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8621176, free=459260, buffers=706740, cached=6388460, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.78, five_min_avg=4.2, fifteen_min_avg=4.3, cpu_share=9.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:15:21.014 7f0bf96c5a20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:15:47.096 7f0bf399b0f0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:15:54.314 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:15:57.856 7f0bf96d2b70 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:16:03.993 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19921344, partition_space_available_1k=61169140, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1432280, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11428, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51644, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8635856, free=435888, buffers=706748, cached=6397144, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.73, fifteen_min_avg=4.13, cpu_share=10.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:16:54.485 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:17:05.375 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20976272, partition_space_available_1k=60114212, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1427108, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11436, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50876, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8690020, free=431384, buffers=692920, cached=6361312, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.46, five_min_avg=3.69, fifteen_min_avg=4.09, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:17:33.639 7f0bf8e8acc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:17:54.329 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:18:03.351 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21037064, partition_space_available_1k=60053420, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1448156, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13388, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50752, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8925980, free=293472, buffers=688924, cached=6267260, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.0, five_min_avg=3.52, fifteen_min_avg=4.0, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:18:54.337 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:19:05.923 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21054044, partition_space_available_1k=60036440, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1460280, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13452, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50628, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9040064, free=340380, buffers=679680, cached=6115512, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.61, five_min_avg=3.98, fifteen_min_avg=4.13, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:19:54.449 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:20:05.970 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21072120, partition_space_available_1k=60018364, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1486776, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13496, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50528, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9132260, free=345432, buffers=676184, cached=6021760, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.02, five_min_avg=4.14, fifteen_min_avg=4.18, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:20:54.672 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:21:06.998 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21103416, partition_space_available_1k=59987068, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13288, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50420, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9204516, free=369480, buffers=670064, cached=5931576, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=3.98, fifteen_min_avg=4.11, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:21:54.443 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:22:05.106 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21139932, partition_space_available_1k=59950552, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1521888, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13380, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50424, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9152948, free=382708, buffers=670108, cached=5969872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.7, fifteen_min_avg=4.01, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:22:54.356 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:23:03.358 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21154596, partition_space_available_1k=59935888, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1538576, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13456, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50428, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9149264, free=380000, buffers=669648, cached=5976724, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.82, five_min_avg=3.89, fifteen_min_avg=4.06, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:23:54.419 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:24:03.892 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21167160, partition_space_available_1k=59923324, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1542712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13456, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50368, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9173648, free=407392, buffers=664680, cached=5929916, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.39, five_min_avg=3.69, fifteen_min_avg=3.97, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:24:54.366 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:25:03.338 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21175648, partition_space_available_1k=59914836, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1555664, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13488, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50372, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9193740, free=373232, buffers=664756, cached=5943908, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.09, five_min_avg=3.55, fifteen_min_avg=3.91, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:25:31.289 7f0bfb7bcc18 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 01/20/2025 02:25:54.479 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:26:03.344 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21192268, partition_space_available_1k=59898216, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1564940, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13604, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50336, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9063656, free=514764, buffers=664168, cached=5933048, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=3.44, fifteen_min_avg=3.84, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:26:54.385 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 01/20/2025 02:27:03.383 7f0bfbde5808 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2650-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21239260, partition_space_available_1k=59851224, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1571396, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13496, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50336, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9014368, free=477220, buffers=667488, cached=6016560, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.99, five_min_avg=3.35, fifteen_min_avg=3.79, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 01/20/2025 02:27:17.036 7f0bfb7bc978 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map