02/17/2025 00:52:54.205 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/17/2025 00:52:54.205 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/17/2025 00:52:54.205 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/17/2025 00:52:54.213 7f46d5815648 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/17/2025 00:52:54.214 7f46d5815648 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/17/2025 00:52:54.214 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/17/2025 00:52:54.215 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 00:52:54.215 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 00:52:54.215 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/17/2025 00:52:54.215 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.216 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 00:52:54.216 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.216 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.216 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/17/2025 00:52:54.216 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.288 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 13270988444300079609000029092360702628890885240277505204776856833503233745115, 'start': '1739753572000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 57074039278535679863856197893024828544367204515222822722432698438746995741343, 'start': '1739753572000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 96641415993912561541366384546708908431857143691548823099858428539871274883743, 'start': '1739753572000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/17/2025 00:52:54.410 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.411 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/17/2025 00:52:54.411 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.411 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753574411463, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:52:54.411 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753574411806, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739753572000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739753572000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739753572000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:52:54.414 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.414 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 00:52:54.414 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.414 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.414 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739753572000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739753572000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739753572000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/17/2025 00:52:54.414 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.716 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:52:54.955 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 00:52:54.957 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:54.957 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. database_usage = [ << disk_space_used_1k = 6722496 disk_space_available_1k = 74367988 analytics_db_size_1k = 108 >>, ] >> 02/17/2025 00:52:54.957 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:55.445 7f46db16ddf0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/17/2025 00:52:55.445 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753575445855, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:52:55.447 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:55.447 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 00:52:55.448 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:57.885 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753577885875, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:52:57.886 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 00:52:57.889 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:57.890 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 02/17/2025 00:52:57.890 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:58.216 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:52:58.217 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:52:58.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:58.219 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:52:58.219 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:58.219 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:52:58.219 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:52:58.219 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:02.218 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:02.220 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:02.220 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:02.220 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:02.220 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:02.220 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:02.220 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:02.221 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:03.052 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753583052177, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739753572000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739753572000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739753572000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:53:03.052 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6723320, partition_space_available_1k=74367164, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1215184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7492, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51500, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4300516, free=5441680, buffers=237636, cached=6195808, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.82, five_min_avg=0.86, fifteen_min_avg=0.43, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:53:03.052 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:03.053 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739753572000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739753572000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739753572000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/17/2025 00:53:03.053 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:03.053 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:03.053 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6723320 partition_space_available_1k = 74367164 percentage_partition_space_used = 8 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1215184 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7492 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51500 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 4300516 free = 5441680 buffers = 237636 cached = 6195808 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.82 five_min_avg = 0.86 fifteen_min_avg = 0.43 cpu_share = 0 node_type = database-node >> >> 02/17/2025 00:53:03.053 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:06.220 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:06.222 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:06.223 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:06.223 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:06.223 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:06.223 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:06.223 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:06.223 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:10.222 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:10.224 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:10.224 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:10.224 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:10.224 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:10.224 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:10.224 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:10.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:14.224 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:14.225 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:14.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:14.225 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:14.226 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:14.227 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:14.227 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:18.227 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:18.229 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:18.229 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:18.229 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:18.229 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:18.229 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:18.229 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:18.230 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:22.232 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:22.234 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:22.234 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:22.234 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:22.234 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:22.234 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:22.234 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:22.235 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:26.235 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:26.236 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:26.236 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:26.236 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:26.237 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:26.238 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:26.238 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:30.236 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:30.238 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:30.238 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:30.238 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:30.238 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:30.238 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:30.239 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:34.242 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:34.243 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:34.243 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:34.243 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:34.243 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:34.243 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:34.244 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:38.247 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:38.248 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:38.248 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:38.249 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:38.249 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:38.249 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:38.249 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:38.249 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:38.249 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:38.250 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:38.250 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:38.250 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:38.250 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:38.250 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:42.250 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:42.252 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:42.252 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:42.252 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:42.252 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:42.252 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:42.252 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:42.253 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:46.255 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:46.257 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 00:53:46.257 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:46.257 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:46.257 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:46.257 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:46.257 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739753574215698 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.22:8086, 10.0.0.53:8086, 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 00:53:46.258 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 00:53:50.260 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 00:53:50.261 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 00:53:50.261 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 00:53:50.262 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:50.262 7f46d5815e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 00:53:50.262 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 00:53:50.262 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 00:53:50.262 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 00:53:50.263 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 00:53:50.263 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 00:53:50.263 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 00:53:54.638 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:53:54.964 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 00:53:55.604 7f46db16ddf0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753635604088, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.53:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:53:55.607 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 00:53:57.929 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753637929185, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=2, pending=2, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:53:57.934 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 00:54:03.057 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6795708, partition_space_available_1k=74294776, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1301628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9400, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51876, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4535020, free=4864836, buffers=250104, cached=6525680, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.37, five_min_avg=0.95, fifteen_min_avg=0.49, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:54:03.058 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 00:54:03.144 7f46d5928348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 02/17/2025 00:54:03.145 7f46d5928348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 00:54:03.146 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/17/2025 00:54:03.146 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/17/2025 00:54:03.146 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 00:54:03.146 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 00:54:03.146 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 00:54:03.146 7f46d5815e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 00:54:03.147 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/17/2025 00:54:03.147 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 00:54:03.148 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 00:54:15.043 7f46d5928348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 02/17/2025 00:54:15.043 7f46d5928348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 00:54:15.044 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/17/2025 00:54:15.044 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 02/17/2025 00:54:15.044 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 00:54:15.044 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 00:54:15.044 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 00:54:15.044 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 00:54:15.045 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 00:54:15.045 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 00:54:54.652 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:54:58.116 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739753698116101, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 00:55:03.140 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7173892, partition_space_available_1k=73916592, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1322844, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10160, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51936, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5515936, free=3698960, buffers=260908, cached=6699836, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.69, five_min_avg=1.41, fifteen_min_avg=0.68, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:55:54.684 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:56:03.084 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7249308, partition_space_available_1k=73841176, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1324728, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10260, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51944, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5523756, free=3610944, buffers=262432, cached=6778508, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.23, five_min_avg=1.52, fifteen_min_avg=0.76, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:56:54.698 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:57:03.203 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7260160, partition_space_available_1k=73830324, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364732, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10328, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51968, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5630840, free=3401928, buffers=264468, cached=6878404, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.94, five_min_avg=1.86, fifteen_min_avg=0.93, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:57:12.272 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 00:57:34.158 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 00:57:54.686 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:58:03.102 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7714924, partition_space_available_1k=73375560, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339104, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10492, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51996, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5652104, free=2992472, buffers=271400, cached=7259664, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.55, five_min_avg=2.31, fifteen_min_avg=1.15, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:58:54.695 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 00:59:03.489 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7723356, partition_space_available_1k=73367128, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10572, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52008, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5714048, free=2916260, buffers=272792, cached=7272540, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.75, five_min_avg=2.29, fifteen_min_avg=1.21, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 00:59:54.701 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:00:03.542 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7724952, partition_space_available_1k=73365532, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1370448, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10572, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52020, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5777596, free=2854788, buffers=273888, cached=7269368, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.68, five_min_avg=2.33, fifteen_min_avg=1.3, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:00:54.757 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:01:03.244 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7734260, partition_space_available_1k=73356224, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353836, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10420, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52032, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5809812, free=2806752, buffers=274668, cached=7284408, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.58, five_min_avg=2.4, fifteen_min_avg=1.39, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:01:54.750 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:02:03.196 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7744832, partition_space_available_1k=73345652, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358972, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10500, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52040, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5814444, free=2795276, buffers=275348, cached=7290572, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.69, five_min_avg=2.46, fifteen_min_avg=1.48, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:02:54.727 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:03:03.191 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7751488, partition_space_available_1k=73338996, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1366628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10548, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52048, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5839444, free=2757744, buffers=275848, cached=7302604, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.49, five_min_avg=2.46, fifteen_min_avg=1.54, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:03:54.770 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:04:03.154 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7762332, partition_space_available_1k=73328152, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1371288, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10620, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52064, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5905208, free=2684288, buffers=276144, cached=7310000, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.14, five_min_avg=2.62, fifteen_min_avg=1.65, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:04:54.751 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:05:03.161 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7768532, partition_space_available_1k=73321952, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1379200, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10652, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5950024, free=2627952, buffers=276452, cached=7321212, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.94, five_min_avg=2.63, fifteen_min_avg=1.71, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:05:54.750 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:06:03.262 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7778880, partition_space_available_1k=73311604, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1384816, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10652, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52076, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6004640, free=2567392, buffers=276684, cached=7326924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.57, five_min_avg=2.59, fifteen_min_avg=1.76, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:06:54.768 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:07:03.196 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7788464, partition_space_available_1k=73302020, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1386792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10684, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6037208, free=2524292, buffers=276884, cached=7337256, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.66, five_min_avg=2.59, fifteen_min_avg=1.81, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:07:54.763 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:08:03.177 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7799828, partition_space_available_1k=73290656, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1401056, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10700, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52096, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6114720, free=2440048, buffers=277012, cached=7343860, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.16, five_min_avg=2.77, fifteen_min_avg=1.93, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:08:54.809 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:09:03.186 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7805680, partition_space_available_1k=73284804, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1395432, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10740, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52100, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6139764, free=2402824, buffers=277088, cached=7355964, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.19, five_min_avg=2.56, fifteen_min_avg=1.91, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:09:54.795 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:10:03.195 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7815412, partition_space_available_1k=73275072, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1398596, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10744, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52096, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6148312, free=2389796, buffers=277196, cached=7360336, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.39, five_min_avg=2.78, fifteen_min_avg=2.02, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:10:54.897 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:11:03.194 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7824856, partition_space_available_1k=73265628, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1404096, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10760, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52104, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6161176, free=2367288, buffers=277256, cached=7369920, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=2.78, fifteen_min_avg=2.07, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:11:54.863 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:12:03.211 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7828424, partition_space_available_1k=73262060, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1408696, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10784, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52108, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6168568, free=2345768, buffers=277300, cached=7384004, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.73, five_min_avg=2.72, fifteen_min_avg=2.09, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:12:54.796 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:13:03.299 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7837816, partition_space_available_1k=73252668, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1415488, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10676, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52108, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6181432, free=2328004, buffers=277320, cached=7388884, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.55, five_min_avg=2.67, fifteen_min_avg=2.11, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:13:54.791 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:14:03.397 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7848928, partition_space_available_1k=73241556, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1418960, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52120, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6202928, free=2293816, buffers=277360, cached=7401536, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.63, five_min_avg=2.68, fifteen_min_avg=2.15, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:14:54.823 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:15:03.205 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7859668, partition_space_available_1k=73230816, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1427608, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10760, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52120, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6215228, free=2271112, buffers=277368, cached=7411932, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.88, five_min_avg=2.71, fifteen_min_avg=2.19, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:15:54.829 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:16:03.207 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7869696, partition_space_available_1k=73220788, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1425380, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10800, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52120, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6229548, free=2250868, buffers=277380, cached=7417844, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=2.71, fifteen_min_avg=2.22, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:16:54.781 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:17:03.221 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7852792, partition_space_available_1k=73237692, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1428660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10800, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52128, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6200628, free=2286864, buffers=277396, cached=7410752, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=2.61, fifteen_min_avg=2.22, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:17:54.803 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:18:03.284 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7863700, partition_space_available_1k=73226784, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1437680, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10836, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52128, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6209764, free=2276024, buffers=277416, cached=7412436, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.62, five_min_avg=2.61, fifteen_min_avg=2.24, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:18:54.809 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:19:03.379 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7869780, partition_space_available_1k=73220704, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1447004, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10732, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6223504, free=2250556, buffers=277424, cached=7424156, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.63, five_min_avg=2.6, fifteen_min_avg=2.26, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:19:54.790 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:20:03.212 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7877040, partition_space_available_1k=73213444, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1444328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10764, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6244060, free=2223496, buffers=277432, cached=7430652, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.0, five_min_avg=2.66, fifteen_min_avg=2.3, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:20:54.798 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:21:03.215 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7885504, partition_space_available_1k=73204980, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1450044, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10776, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6237784, free=2220792, buffers=277436, cached=7439628, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.36, five_min_avg=2.54, fifteen_min_avg=2.28, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:21:54.807 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:22:03.309 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7893344, partition_space_available_1k=73197140, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1454112, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10792, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52152, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6254268, free=2196444, buffers=277444, cached=7447484, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=2.44, fifteen_min_avg=2.26, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:22:54.800 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:23:03.214 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7901148, partition_space_available_1k=73189336, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1458384, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10828, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6266700, free=2175884, buffers=277448, cached=7455608, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.4, five_min_avg=2.44, fifteen_min_avg=2.27, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:23:54.824 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:24:03.251 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7904844, partition_space_available_1k=73185640, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1471556, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52176, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6277652, free=2155156, buffers=277452, cached=7465380, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.63, five_min_avg=2.51, fifteen_min_avg=2.31, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:24:54.810 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:25:03.236 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7917656, partition_space_available_1k=73172828, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1469848, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10788, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52512, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6287132, free=2138772, buffers=277452, cached=7472284, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.57, five_min_avg=2.51, fifteen_min_avg=2.32, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:25:54.841 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:26:03.244 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7924352, partition_space_available_1k=73166132, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1475244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10800, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52516, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6300528, free=2122760, buffers=277456, cached=7474896, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=2.49, fifteen_min_avg=2.32, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:26:54.880 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:27:03.269 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7931916, partition_space_available_1k=73158568, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1481032, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10812, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52516, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6315904, free=2099820, buffers=277456, cached=7482460, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.37, five_min_avg=2.46, fifteen_min_avg=2.32, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:27:54.883 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:28:03.270 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7934032, partition_space_available_1k=73156452, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1485480, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52520, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6319972, free=2088652, buffers=277456, cached=7489560, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.27, five_min_avg=2.42, fifteen_min_avg=2.32, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:28:54.857 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:29:03.291 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7944964, partition_space_available_1k=73145520, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1490648, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10856, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52524, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6321008, free=2080560, buffers=277456, cached=7496616, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.36, five_min_avg=2.41, fifteen_min_avg=2.32, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:29:54.866 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:30:03.284 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7946044, partition_space_available_1k=73144440, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1495268, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52520, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6331800, free=2064592, buffers=277456, cached=7501792, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=2.43, fifteen_min_avg=2.33, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:30:54.885 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:31:03.314 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7951584, partition_space_available_1k=73138900, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1499064, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52528, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6338704, free=2047092, buffers=277464, cached=7512380, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.47, five_min_avg=2.43, fifteen_min_avg=2.33, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:31:54.915 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:32:03.379 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7957728, partition_space_available_1k=73132756, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503820, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10824, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52532, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6344380, free=2035200, buffers=277464, cached=7518596, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=2.44, fifteen_min_avg=2.35, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:32:54.919 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:33:03.321 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7969556, partition_space_available_1k=73120928, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1509356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10840, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52536, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6350744, free=2026168, buffers=277464, cached=7521264, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.37, five_min_avg=2.42, fifteen_min_avg=2.35, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:33:54.918 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:34:03.321 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7977216, partition_space_available_1k=73113268, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1513556, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10856, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52540, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6353016, free=2015216, buffers=277468, cached=7529940, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=2.72, fifteen_min_avg=2.46, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:34:54.916 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:35:03.334 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7984964, partition_space_available_1k=73105520, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1519712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10884, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52540, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6368228, free=1988368, buffers=277468, cached=7541576, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=2.63, fifteen_min_avg=2.44, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:35:54.926 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:36:03.347 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7999528, partition_space_available_1k=73090956, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1492828, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10948, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52544, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6319152, free=2016776, buffers=277468, cached=7562244, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.72, five_min_avg=2.67, fifteen_min_avg=2.47, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:36:54.931 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:37:03.359 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8012440, partition_space_available_1k=73078044, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1493544, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10836, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52548, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6314824, free=2018048, buffers=277468, cached=7565300, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.92, five_min_avg=2.72, fifteen_min_avg=2.49, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:37:54.957 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:38:03.478 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8021364, partition_space_available_1k=73069120, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1499080, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10856, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52548, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6317928, free=2011164, buffers=277468, cached=7569080, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.15, five_min_avg=2.8, fifteen_min_avg=2.54, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:38:54.956 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:39:03.365 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8024500, partition_space_available_1k=73065984, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503528, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10864, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52552, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6330820, free=1989404, buffers=277468, cached=7577948, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.64, five_min_avg=2.7, fifteen_min_avg=2.52, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:39:55.010 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:40:03.426 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8028252, partition_space_available_1k=73062232, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1508240, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52556, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6318324, free=1993724, buffers=277468, cached=7586124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.74, five_min_avg=2.72, fifteen_min_avg=2.54, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:40:54.959 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:41:03.379 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8036228, partition_space_available_1k=73054256, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1513808, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10916, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52560, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6341472, free=1962420, buffers=277468, cached=7594280, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.72, five_min_avg=2.72, fifteen_min_avg=2.55, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:41:54.952 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:42:03.383 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8038928, partition_space_available_1k=73051556, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1519052, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10936, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52560, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6345808, free=1950344, buffers=277468, cached=7602020, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.86, five_min_avg=2.76, fifteen_min_avg=2.57, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:42:55.011 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:43:03.391 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8050000, partition_space_available_1k=73040484, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1522700, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10900, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52564, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6361576, free=1927860, buffers=277468, cached=7608736, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=2.77, fifteen_min_avg=2.59, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:43:54.981 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:44:03.461 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8057444, partition_space_available_1k=73033040, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1527344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10904, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52568, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6363892, free=1917504, buffers=277468, cached=7616776, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=2.74, fifteen_min_avg=2.59, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:44:54.974 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:45:03.389 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8062812, partition_space_available_1k=73027672, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1532172, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10916, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52568, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6367272, free=1909500, buffers=277468, cached=7621400, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.5, five_min_avg=2.7, fifteen_min_avg=2.58, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:45:55.004 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:46:03.398 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8067920, partition_space_available_1k=73022564, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1535804, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10920, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52572, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6371092, free=1899924, buffers=277468, cached=7627156, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.98, five_min_avg=2.79, fifteen_min_avg=2.62, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:46:55.001 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:47:03.397 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8075364, partition_space_available_1k=73015120, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1544220, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10964, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52576, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6377528, free=1886180, buffers=277468, cached=7634464, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.48, five_min_avg=2.68, fifteen_min_avg=2.59, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:47:54.965 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:48:03.400 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8087568, partition_space_available_1k=73002916, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1551048, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10984, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52576, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6388692, free=1867488, buffers=277468, cached=7641992, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.7, five_min_avg=2.7, fifteen_min_avg=2.61, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:48:54.984 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:49:03.406 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8090224, partition_space_available_1k=73000260, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1553160, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10860, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6389476, free=1857268, buffers=277472, cached=7651424, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.81, five_min_avg=2.72, fifteen_min_avg=2.62, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:49:54.982 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:50:03.449 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8098820, partition_space_available_1k=72991664, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1558944, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6378416, free=1861280, buffers=277488, cached=7658456, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=2.74, fifteen_min_avg=2.64, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:50:54.987 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:51:03.397 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8105300, partition_space_available_1k=72985184, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1563036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10900, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52588, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6399284, free=1828520, buffers=277488, cached=7670348, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=2.73, fifteen_min_avg=2.64, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:51:54.983 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:52:03.423 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8112472, partition_space_available_1k=72978012, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1567024, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10944, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52592, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6404164, free=1825920, buffers=277488, cached=7668068, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.84, five_min_avg=2.77, fifteen_min_avg=2.66, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:52:54.983 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:53:03.404 7f46db16ddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8119188, partition_space_available_1k=72971296, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1573860, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52596, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6417396, free=1806612, buffers=277500, cached=7674132, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.48, five_min_avg=2.69, fifteen_min_avg=2.64, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:53:23.833 7f46d5928348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:53:23.835 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/17/2025 01:53:23.844 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:53:23.846 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/17/2025 01:53:23.846 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:53:23.846 7f46d4ab6048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/17/2025 01:53:23.846 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:53:23.846 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:53:23.847 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:53:23.847 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:53:23.847 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:23.847 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:24.031 7f46d5928348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/17/2025 01:53:24.031 7f46d5928348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 01:53:24.032 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/17/2025 01:53:24.032 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/17/2025 01:53:24.032 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 01:53:24.032 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 01:53:24.032 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 01:53:24.032 7f46d4ab6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 01:53:24.033 7f46d4ab6048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/17/2025 01:53:24.033 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 01:53:24.034 7f46d5815e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 01:53:41.432 7eff25877f20 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/17/2025 01:53:41.432 7eff25877f20 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/17/2025 01:53:41.433 7eff25877f20 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/17/2025 01:53:41.441 7eff1fc97748 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/17/2025 01:53:41.442 7eff1fc97748 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/17/2025 01:53:41.443 7eff1e8ae048 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/17/2025 01:53:41.460 7eff1e8ae048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/17/2025 01:53:41.461 7eff1e8ae048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:41.910 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 13270988444300079609000029092360702628890885240277505204776856833503233745115, 'start': '1739757217000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 57074039278535679863856197893024828544367204515222822722432698438746995741343, 'start': '1739757217000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 96641415993912561541366384546708908431857143691548823099858428539871274883743, 'start': '1739757217000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/17/2025 01:53:42.334 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:42.335 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/17/2025 01:53:42.335 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:42.335 7eff25877f20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757222335504, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:53:42.335 7eff25877f20 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757222335821, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:53:42.368 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:42.368 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 01:53:42.368 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:42.368 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:42.369 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739757217000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739757217000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739757217000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/17/2025 01:53:42.369 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:43.588 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:53:45.278 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 01:53:45.284 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:45.284 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. database_usage = [ << disk_space_used_1k = 7990836 disk_space_available_1k = 73099648 analytics_db_size_1k = 29604 >>, ] >> 02/17/2025 01:53:45.284 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:45.475 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:53:45.491 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:53:45.491 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:53:45.491 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:53:45.492 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:53:45.492 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:53:45.492 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:53:45.508 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:53:45.509 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:53:45.509 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:45.509 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:47.527 7eff25877f20 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/17/2025 01:53:47.527 7eff25877f20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757227527254, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.53:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:53:47.529 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:51.197 7eff1e86e448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:53:51.197 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/17/2025 01:53:51.198 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/17/2025 01:53:51.198 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:51.198 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/17/2025 01:53:51.198 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:51.199 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:51.199 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/17/2025 01:53:51.199 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:53:53.387 7eff25877f20 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 02/17/2025 01:53:55.199 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:53:55.201 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:53:55.201 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:53:55.201 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:53:55.201 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:53:55.202 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:53:55.202 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:53:55.202 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:53:55.202 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:53:55.202 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:55.202 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:53:55.207 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/17/2025 01:53:55.208 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 01:53:55.208 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/17/2025 01:53:55.208 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/17/2025 01:53:55.209 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 01:53:55.209 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 01:53:55.209 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 01:53:55.209 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/17/2025 01:53:55.210 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 01:53:55.211 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 01:53:58.960 7eff25877f20 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757238960358, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:53:58.961 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8116704, partition_space_available_1k=72973780, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1204744, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9264, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44392, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4967088, free=3160684, buffers=281780, cached=7766088, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.77, five_min_avg=4.17, fifteen_min_avg=3.13, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:54:42.700 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:54:44.613 7eff25877f20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757284612932, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.56:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:54:49.784 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757289784189, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=1), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:54:49.784 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 01:54:54.883 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=9280768, partition_space_available_1k=71809716, percentage_partition_space_used=11, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1322356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9920, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44420, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5404260, free=1491324, buffers=352740, cached=8927316, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.33, five_min_avg=5.42, fifteen_min_avg=3.62, cpu_share=21.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:55:42.777 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:55:54.829 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10143880, partition_space_available_1k=70946604, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1320920, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10020, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44440, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5750472, free=281624, buffers=382404, cached=9761140, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.65, five_min_avg=5.5, fifteen_min_avg=3.76, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:56:24.315 7eff1e86e448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:56:24.316 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/17/2025 01:56:24.317 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:56:24.318 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/17/2025 01:56:24.318 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:24.318 7eff1e8ae148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/17/2025 01:56:24.318 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:24.318 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:56:24.318 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:56:24.319 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:56:24.319 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:56:24.319 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:56:24.417 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/17/2025 01:56:24.418 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 01:56:24.418 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/17/2025 01:56:24.419 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/17/2025 01:56:24.419 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 01:56:24.420 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 01:56:24.420 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 01:56:24.420 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 01:56:24.421 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/17/2025 01:56:24.422 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 01:56:24.426 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 01:56:24.450 7eff1e86e448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:56:24.451 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/17/2025 01:56:24.452 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:56:24.452 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/17/2025 01:56:24.452 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:24.453 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/17/2025 01:56:24.453 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:24.453 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:56:24.453 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:56:24.454 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:56:24.454 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:56:24.454 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:56:24.477 7eff1e86e448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:56:24.492 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/17/2025 01:56:24.492 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/17/2025 01:56:24.493 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:24.493 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/17/2025 01:56:24.493 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:24.493 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:24.494 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:24.494 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:28.496 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:56:28.498 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:56:28.498 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:56:28.498 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:28.498 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:56:28.498 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:28.499 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:56:28.499 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:56:28.499 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:56:28.499 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:56:28.499 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:56:36.890 7eff1e86e448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:56:36.890 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:36.891 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:40.892 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:56:40.894 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:56:40.894 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:56:40.894 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:40.895 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:56:40.895 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:40.895 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:56:40.896 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:56:40.896 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:40.896 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:56:40.897 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:40.897 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:40.897 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:40.897 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:42.577 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:56:42.847 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:42.848 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. database_usage = [ << disk_space_used_1k = 10393948 disk_space_available_1k = 70696536 analytics_db_size_1k = 29680 >>, ] >> 02/17/2025 01:56:42.848 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:44.894 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:56:44.896 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:56:44.897 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:56:44.899 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:44.900 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:56:44.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:44.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:56:44.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:56:44.901 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:44.901 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:56:44.901 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:44.901 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:44.901 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:44.902 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:48.897 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:56:48.899 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:56:48.899 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:56:48.899 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:48.899 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:56:48.899 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:48.899 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:48.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:50.751 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10393980, partition_space_available_1k=70696504, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1326292, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10784, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44476, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4320928, free=1746696, buffers=388516, cached=9719500, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.18, five_min_avg=4.92, fifteen_min_avg=3.67, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:56:50.751 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:50.752 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10393980 partition_space_available_1k = 70696504 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1326292 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 10784 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 44476 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 4320928 free = 1746696 buffers = 388516 cached = 9719500 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.18 five_min_avg = 4.92 fifteen_min_avg = 3.67 cpu_share = 7.75 node_type = database-node >> >> 02/17/2025 01:56:50.752 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:52.900 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:56:52.902 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:56:52.902 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:56:52.902 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:52.902 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:56:52.902 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:52.902 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:52.903 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:56.904 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:56:56.906 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:56:56.906 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:56:56.906 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:56.906 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:56:56.906 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:56:56.906 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:56:56.907 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:56:56.907 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:56.907 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:56:56.907 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:56.907 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:56:56.907 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:56:56.908 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:00.907 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:00.908 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:00.909 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:00.910 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:00.910 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:00.910 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:00.910 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:00.910 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:04.912 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:04.914 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:04.914 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:04.914 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:04.914 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:04.914 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:04.915 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:08.918 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:08.919 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:08.920 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:08.920 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:08.920 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:08.920 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:08.920 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:08.921 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:12.923 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:12.924 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:12.925 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:12.926 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:12.926 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:12.926 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:12.926 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:12.926 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:16.926 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:16.928 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:16.928 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:16.928 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:16.928 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:16.928 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:16.929 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:16.929 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:16.929 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:16.929 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:16.929 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:16.930 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:16.930 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:16.930 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:20.929 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:20.931 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:20.931 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:20.931 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:20.931 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:20.931 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:20.932 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:20.932 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:20.932 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:20.932 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:20.933 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:20.933 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:20.933 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:20.933 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:24.932 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:24.934 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:24.934 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:24.934 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:24.934 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:24.934 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:24.934 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:24.935 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:28.935 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:28.937 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:28.937 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:28.937 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:28.937 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:28.937 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:28.937 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:28.938 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:32.938 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:32.939 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:32.940 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:32.940 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:32.940 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:32.940 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:32.940 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:32.941 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:36.940 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:36.942 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:36.942 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:36.942 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:36.942 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:36.942 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:36.942 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:36.943 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:40.944 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:40.946 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:40.946 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:40.946 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:40.946 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:40.947 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:40.947 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:40.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:40.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:40.948 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:40.949 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:40.949 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:40.949 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:40.949 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:42.722 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:57:43.103 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:43.104 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. database_usage = [ << disk_space_used_1k = 10318348 disk_space_available_1k = 70772136 analytics_db_size_1k = 29680 >>, ] >> 02/17/2025 01:57:43.104 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:44.946 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:44.947 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:44.947 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:44.947 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:44.947 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:44.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:44.949 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:44.949 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:48.948 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:48.950 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:48.950 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:48.950 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:48.950 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:48.950 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:48.951 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:48.952 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:52.775 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10324864, partition_space_available_1k=70765620, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1328756, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11144, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44564, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4672832, free=1347464, buffers=392720, cached=9762624, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.4, five_min_avg=4.19, fifteen_min_avg=3.5, cpu_share=5.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:57:52.775 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:52.775 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10324864 partition_space_available_1k = 70765620 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1328756 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 11144 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 44564 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 4672832 free = 1347464 buffers = 392720 cached = 9762624 node_type = database-node >> system_cpu_usage = << one_min_avg = 2.4 five_min_avg = 4.19 fifteen_min_avg = 3.5 cpu_share = 5.75 node_type = database-node >> >> 02/17/2025 01:57:52.775 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:52.951 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:52.952 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:52.953 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:52.954 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:52.954 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:52.954 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:52.954 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:56.956 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:57:56.957 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:57:56.957 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:56.958 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:57:56.959 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:56.959 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:57:56.959 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:57:56.959 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:00.958 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:00.960 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:00.960 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:00.960 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:00.960 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:00.960 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:00.961 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:04.960 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:04.962 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:04.962 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:04.962 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:04.962 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:04.962 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:04.962 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:04.963 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:08.963 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:08.964 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:08.964 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:08.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:08.966 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:08.966 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:12.965 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:12.967 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:12.968 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:12.968 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:12.968 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:12.968 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:12.968 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:12.968 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:16.967 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:16.969 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:16.970 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:16.971 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:16.971 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:16.971 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:16.971 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:16.971 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:20.972 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:20.973 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:20.973 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:20.974 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:20.975 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:20.975 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:24.978 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:24.981 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:24.982 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:24.982 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:24.982 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:24.982 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:24.982 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:24.983 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:28.982 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:28.984 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/17/2025 01:58:28.984 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:28.984 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:28.984 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.22:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:28.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:28.986 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.22:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:28.986 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:32.985 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:32.986 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:58:32.986 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:32.986 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:32.986 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:32.986 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:32.986 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:58:32.987 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:58:32.987 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:58:32.987 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:58:32.987 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:58:33.770 7eff1e86e448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/17/2025 01:58:33.770 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1739757221461028 collector_name = collector_ip = 10.0.0.53:8086 collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/17/2025 01:58:33.771 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 01:58:37.772 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 01:58:37.774 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 01:58:37.775 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 01:58:37.775 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:58:37.775 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 01:58:37.785 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/17/2025 01:58:37.786 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 01:58:37.787 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/17/2025 01:58:37.787 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/17/2025 01:58:37.788 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 01:58:37.788 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 01:58:37.788 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 01:58:37.789 7eff1e8ae448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 01:58:37.789 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/17/2025 01:58:37.790 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 01:58:37.791 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 01:58:42.730 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:58:42.789 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 02/17/2025 01:58:42.790 7eff1e86e448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 01:58:42.792 7eff1e8ae148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/17/2025 01:58:42.794 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 02/17/2025 01:58:42.795 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 01:58:42.795 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 01:58:42.795 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 01:58:42.795 7eff1e8ae148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 01:58:42.796 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 01:58:42.796 7eff1e8ae448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 01:58:54.223 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10459148, partition_space_available_1k=70631336, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337296, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11524, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44716, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6032688, free=316684, buffers=396860, cached=9429408, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.01, five_min_avg=4.0, fifteen_min_avg=3.47, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 01:59:42.648 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 01:59:46.656 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757586655935, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=43, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=2), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 01:59:52.197 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10618412, partition_space_available_1k=70472072, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353532, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11576, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44752, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5964916, free=235884, buffers=405884, cached=9568956, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=3.67, fifteen_min_avg=3.39, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:00:42.640 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:00:45.776 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757645775926, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=2), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:00:51.738 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10894600, partition_space_available_1k=70195884, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358404, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11044, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44800, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5947560, free=267060, buffers=408048, cached=9552972, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.41, five_min_avg=3.41, fifteen_min_avg=3.31, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:01:42.671 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:01:51.749 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10894936, partition_space_available_1k=70195548, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1383540, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11092, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44800, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5938996, free=275052, buffers=408068, cached=9553524, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.55, five_min_avg=2.98, fifteen_min_avg=3.17, cpu_share=7.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:02:42.664 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:02:51.785 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10899816, partition_space_available_1k=70190668, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1367380, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11980, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44808, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5996428, free=300932, buffers=407176, cached=9471104, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.49, five_min_avg=2.7, fifteen_min_avg=3.06, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:03:42.658 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:03:54.784 7eff25877f20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11784136, partition_space_available_1k=69306348, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1371300, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12104, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44844, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5584472, free=432612, buffers=427064, cached=9731492, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.34, five_min_avg=2.89, fifteen_min_avg=3.09, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:04:03.854 7eff1e86e248 [contrail-database-nodemgr] [DEBUG]: DBG: event_process_state:contrail-query-engine,group:contrail-query-engine,state:PROCESS_STATE_EXITED 02/17/2025 02:04:03.855 7eff1e86e248 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757843855396, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1739757217000000', last_stop_time='', last_exit_time='1739757843855314', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757217000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-13 01:16:56.444420", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7771-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:04:33.607 7f780f95b688 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/17/2025 02:04:33.607 7f780f95b688 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/17/2025 02:04:33.607 7f780f95b688 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/17/2025 02:04:33.616 7f780d049a20 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/17/2025 02:04:33.616 7f780d049a20 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/17/2025 02:04:33.617 7f780d04c390 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/17/2025 02:04:33.618 7f780d04c390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 02:04:33.619 7f780d04c390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 02:04:33.619 7f780d04c390 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/17/2025 02:04:33.619 7f780d04c390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.619 7f780d04c390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 02:04:33.619 7f780d04c390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.619 7f780d04c390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.620 7f780d04c390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1739757873619129 collector_name = collector_ip = collector_list = [ 10.0.0.53:8086, 10.0.0.56:8086, 10.0.0.22:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/17/2025 02:04:33.620 7f780d04c390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.730 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 32384254260761181615332085248895162433761811865503499861221579152090763055770, 'start': '1739757871000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 98240306331097230776212649351989189332458571351766895988534206471756578805619, 'start': '1739757871000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 115480377674401105483033453726843079874910052328053291961202550557759638818661, 'start': '1739757871000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/17/2025 02:04:33.869 7f7815bb8a20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.870 7f7815bb8a20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-17 00:48:11.717672", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7784-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 02/17/2025 02:04:33.870 7f7815bb8a20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.870 7f780f95b688 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757873870584, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:04:33.870 7f780f95b688 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757873870894, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757871000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757871000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757871000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-17 00:48:11.717672", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7784-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:04:33.872 7f780d062d68 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.873 7f780d062d68 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 02:04:33.873 7f780d062d68 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.873 7f780d062d68 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:33.873 7f780d062d68 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739757871000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739757871000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1739757871000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-17 00:48:11.717672", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7784-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 02/17/2025 02:04:33.873 7f780d062d68 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:34.314 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:04:34.705 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 02:04:34.706 7f780d4bc588 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:34.706 7f780d4bc588 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. database_usage = [ << disk_space_used_1k = 12096232 disk_space_available_1k = 68994252 analytics_db_size_1k = 43564 >>, ] >> 02/17/2025 02:04:34.706 7f780d4bc588 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:35.266 7f780f95b688 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/17/2025 02:04:35.266 7f780f95b688 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757875266592, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:04:35.269 7f780d031d68 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:35.269 7f780d031d68 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/17/2025 02:04:35.270 7f780d031d68 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/17/2025 02:04:37.051 7f780f95b688 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 02/17/2025 02:04:37.619 7f780d547240 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/17/2025 02:04:37.620 7f780d547240 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/17/2025 02:04:37.621 7f780d547240 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/17/2025 02:04:37.621 7f780d547240 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/17/2025 02:04:37.621 7f780d547240 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2738-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.53:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/17/2025 02:04:37.621 7f780d547240 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/17/2025 02:04:37.621 7f780d547240 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/17/2025 02:04:37.621 7f780d547240 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/17/2025 02:04:37.622 7f780d547240 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/17/2025 02:04:37.622 7f780d547240 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 02:04:37.622 7f780d547240 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/17/2025 02:04:37.626 7f780d01bac8 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/17/2025 02:04:37.627 7f780d01bac8 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/17/2025 02:04:37.628 7f780d02b390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/17/2025 02:04:37.628 7f780d02b390 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/17/2025 02:04:37.629 7f780d02b390 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/17/2025 02:04:37.629 7f780d02b390 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 02:04:37.629 7f780d02b390 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/17/2025 02:04:37.629 7f780d02b390 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/17/2025 02:04:37.630 7f780d01b0f0 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/17/2025 02:04:37.631 7f780d01b0f0 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/17/2025 02:04:42.726 7f780f95b688 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757882726216, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757871000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757871000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739757871000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-17 00:48:11.717672", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7784-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:04:42.726 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12096756, partition_space_available_1k=68993728, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1223848, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9720, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51980, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5476624, free=439152, buffers=432976, cached=9826888, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=3.02, fifteen_min_avg=3.13, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:05:34.100 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:05:35.089 7f780f95b688 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757935089827, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.53:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:05:37.297 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2738-2.', _node_type='Database', _instance_id='0', _timestamp=1739757937296958, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/17/2025 02:05:37.297 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/17/2025 02:05:42.749 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12105324, partition_space_available_1k=68985160, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1304092, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9940, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5570980, free=338352, buffers=432988, cached=9833320, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.96, five_min_avg=2.66, fifteen_min_avg=3.0, cpu_share=8.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:06:34.143 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:06:42.744 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12106520, partition_space_available_1k=68983964, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1315792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10268, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52092, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5948060, free=269608, buffers=429792, cached=9528180, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.31, five_min_avg=2.36, fifteen_min_avg=2.87, cpu_share=8.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:07:34.124 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:07:42.759 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12105740, partition_space_available_1k=68984744, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321492, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10320, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52092, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5969016, free=256264, buffers=430100, cached=9520260, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.85, five_min_avg=2.04, fifteen_min_avg=2.73, cpu_share=7.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:08:34.136 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:08:42.769 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12109408, partition_space_available_1k=68981076, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1329428, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10404, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52088, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5994484, free=240092, buffers=430232, cached=9510832, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.78, five_min_avg=1.81, fifteen_min_avg=2.6, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:09:34.150 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:09:42.768 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12116080, partition_space_available_1k=68974404, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346900, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10512, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52072, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6026296, free=257624, buffers=430172, cached=9461548, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.84, five_min_avg=1.64, fifteen_min_avg=2.49, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:10:34.170 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:10:42.787 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12121108, partition_space_available_1k=68969376, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342052, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10484, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6055172, free=224596, buffers=430172, cached=9465700, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.05, five_min_avg=1.56, fifteen_min_avg=2.41, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:11:34.168 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:11:42.798 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12125224, partition_space_available_1k=68965260, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1349216, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10556, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52076, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6041412, free=260104, buffers=430168, cached=9443956, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.06, five_min_avg=1.49, fifteen_min_avg=2.33, cpu_share=6.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:12:34.177 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:12:42.808 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12129104, partition_space_available_1k=68961380, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376264, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10584, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6066032, free=250548, buffers=430168, cached=9428892, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.96, five_min_avg=1.37, fifteen_min_avg=2.23, cpu_share=6.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:13:34.190 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:13:42.808 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12133340, partition_space_available_1k=68957144, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364872, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10660, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52072, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6064452, free=273440, buffers=430168, cached=9407580, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.8, five_min_avg=1.49, fifteen_min_avg=2.22, cpu_share=6.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:14:34.216 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:14:42.821 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12127188, partition_space_available_1k=68963296, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1365780, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10752, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52072, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6064344, free=280152, buffers=430168, cached=9400976, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.11, five_min_avg=1.35, fifteen_min_avg=2.12, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:15:34.199 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:15:42.838 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12132380, partition_space_available_1k=68958104, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377372, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10800, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6091480, free=248616, buffers=430168, cached=9405376, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.19, five_min_avg=1.31, fifteen_min_avg=2.06, cpu_share=6.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:16:34.209 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:16:42.819 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12136608, partition_space_available_1k=68953876, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1387048, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10752, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52080, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6099532, free=251084, buffers=430168, cached=9394856, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.87, five_min_avg=1.19, fifteen_min_avg=1.97, cpu_share=6.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:17:34.242 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:17:42.829 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12140520, partition_space_available_1k=68949964, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1392136, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10780, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52060, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6107488, free=286068, buffers=429884, cached=9352200, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.45, five_min_avg=1.28, fifteen_min_avg=1.95, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:18:34.232 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:18:42.851 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12144848, partition_space_available_1k=68945636, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1392592, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10800, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6108288, free=279056, buffers=429956, cached=9358340, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.12, five_min_avg=1.22, fifteen_min_avg=1.89, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:19:34.222 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:19:42.847 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12148540, partition_space_available_1k=68941944, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1395896, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52076, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6105548, free=276968, buffers=429956, cached=9363168, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.9, five_min_avg=1.13, fifteen_min_avg=1.81, cpu_share=6.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:20:32.757 7f780d53bf60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 02:20:34.235 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:20:42.857 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12508296, partition_space_available_1k=68582188, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1406176, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10940, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52000, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6123456, free=283160, buffers=434616, cached=9334408, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.29, five_min_avg=1.18, fifteen_min_avg=1.79, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:20:47.735 7f780d53b4e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 02:21:34.241 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:21:42.851 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12515016, partition_space_available_1k=68575468, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1408304, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52004, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6113256, free=287924, buffers=434648, cached=9339812, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.06, five_min_avg=1.13, fifteen_min_avg=1.73, cpu_share=6.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:22:14.853 7f7810babc18 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 02:22:34.238 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:22:42.850 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12512120, partition_space_available_1k=68578364, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1439468, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11908, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51996, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6148596, free=265776, buffers=434472, cached=9326796, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.24, five_min_avg=1.17, fifteen_min_avg=1.7, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:23:34.254 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:23:42.870 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12525124, partition_space_available_1k=68565360, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1450036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12672, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51992, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6208544, free=240312, buffers=433884, cached=9292900, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.82, five_min_avg=1.69, fifteen_min_avg=1.85, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:24:34.346 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:24:42.873 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12542880, partition_space_available_1k=68547604, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1478436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12776, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51968, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6238420, free=278632, buffers=433824, cached=9224764, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=1.88, fifteen_min_avg=1.9, cpu_share=10.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:25:34.273 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:25:42.881 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12556444, partition_space_available_1k=68534040, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1489376, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12832, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51964, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6229768, free=284568, buffers=433812, cached=9227492, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=1.81, fifteen_min_avg=1.88, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:26:34.270 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:26:42.917 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12570252, partition_space_available_1k=68520232, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503776, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12840, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51960, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6279448, free=257892, buffers=433816, cached=9204484, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.92, five_min_avg=1.8, fifteen_min_avg=1.87, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:27:34.270 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:27:42.873 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12575272, partition_space_available_1k=68515212, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1520412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12872, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51952, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6266444, free=294392, buffers=433784, cached=9181020, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.53, five_min_avg=1.7, fifteen_min_avg=1.83, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:28:34.287 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:28:42.868 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12580704, partition_space_available_1k=68509780, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1527724, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12660, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51952, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6342248, free=221956, buffers=433760, cached=9177676, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.54, five_min_avg=1.71, fifteen_min_avg=1.83, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:29:34.282 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:29:39.917 7f780d204048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/17/2025 02:29:42.882 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12609056, partition_space_available_1k=68481428, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1658608, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12792, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6387576, free=260852, buffers=434000, cached=9093212, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.55, five_min_avg=1.72, fifteen_min_avg=1.82, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:30:34.291 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:30:42.880 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2738-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12612476, partition_space_available_1k=68478008, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1662132, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6367484, free=277332, buffers=433976, cached=9096848, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.28, five_min_avg=1.62, fifteen_min_avg=1.78, cpu_share=8.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/17/2025 02:31:34.300 7f780f95b688 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/17/2025 02:31:34.395 7f7807cba390 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map