02/06/2025 00:50:55.243 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 00:50:55.243 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 00:50:55.243 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 00:50:55.256 7f04ac369648 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/06/2025 00:50:55.257 7f04ac369648 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/06/2025 00:50:55.258 7f04ac369e48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 00:50:55.262 7f04ac369e48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 00:50:55.263 7f04ac369e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.557 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 54819549615629401826731235229611083437078519848092022032527359883935528098556, 'start': '1738803052000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 37649660072448418711558812740284777635319678657638139363427579531688922968974, 'start': '1738803052000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 88402971442217857948901169469469850781968913809591291432142960925096129828939, 'start': '1738803052000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 00:50:55.801 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.802 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/06/2025 00:50:55.802 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.803 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803055803263, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:55.803 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803055803553, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:55.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.806 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:50:55.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.806 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 00:50:55.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:56.234 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:50:57.069 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 00:50:57.071 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:57.071 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 13790304 disk_space_available_1k = 67300180 analytics_db_size_1k = 4 >>, ] >> 02/06/2025 00:50:57.071 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:57.816 7f04b1cb3df0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 00:50:57.816 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803057816238, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:57.818 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:57.818 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:50:57.818 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:59.258 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:59.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:59.264 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:59.264 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:59.264 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:59.264 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:59.264 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:59.264 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.250 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803062250169, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=1, pending=1, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:51:02.250 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 00:51:02.252 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.253 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 1 pending = 1 all_time_blocked = 0 >>, ] >> 02/06/2025 00:51:02.253 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:03.264 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:03.265 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:03.265 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:03.265 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:03.265 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:03.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:03.267 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:03.267 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:04.138 7f04b1cb3df0 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1 02/06/2025 00:51:07.266 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:07.268 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:07.268 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:07.268 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:07.268 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:07.268 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:07.268 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:07.269 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:09.644 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803069643996, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:51:09.644 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13793248, partition_space_available_1k=67297236, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1231724, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7572, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51876, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7449456, free=427928, buffers=651100, cached=7647152, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.59, five_min_avg=4.26, fifteen_min_avg=2.43, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:51:09.644 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:09.644 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 00:51:09.645 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:09.645 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:09.645 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13793248 partition_space_available_1k = 67297236 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1231724 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7572 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51876 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7449456 free = 427928 buffers = 651100 cached = 7647152 node_type = database-node >> system_cpu_usage = << one_min_avg = 5.59 five_min_avg = 4.26 fifteen_min_avg = 2.43 cpu_share = 0 node_type = database-node >> >> 02/06/2025 00:51:09.645 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:11.268 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:11.270 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:11.270 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:11.270 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:11.271 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:11.271 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:11.271 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:11.271 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:11.271 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:11.271 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:11.272 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:11.272 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:11.272 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:11.272 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:15.272 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:15.273 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:15.273 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:15.273 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:15.273 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:15.274 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:15.275 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:19.276 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:19.277 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:19.277 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:19.278 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:19.278 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:19.278 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:19.278 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:19.279 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:19.279 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:19.279 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:19.279 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:19.279 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:19.279 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:19.280 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:23.280 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:23.281 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:23.281 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:23.281 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:23.281 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:23.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:23.283 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:23.283 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:27.282 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:27.283 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:27.283 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:27.283 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:27.283 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:27.283 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:27.284 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:27.285 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:31.284 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:31.285 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:31.285 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:31.285 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:31.285 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:31.286 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:31.286 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:31.286 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:31.286 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:31.287 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:31.287 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:31.287 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:31.287 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:31.287 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:35.288 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:35.289 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:35.289 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:35.289 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:35.290 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:35.291 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:35.291 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:39.292 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:39.293 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:39.293 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:39.293 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:39.294 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:39.294 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:39.294 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:39.294 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:39.294 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:39.295 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:39.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:39.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:39.295 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:39.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:43.293 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:43.294 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:43.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:43.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:43.295 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:43.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:43.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:43.295 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:43.296 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:43.296 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:43.296 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:43.296 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:43.296 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:43.296 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:47.297 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:47.298 7f04ab639348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 00:51:47.298 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:47.298 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:47.299 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:47.299 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:47.299 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 00:51:47.299 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 00:51:47.300 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 00:51:47.300 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:47.300 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:56.179 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:51:56.642 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:57.452 7f04b1cb3df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803117452181, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:51:57.454 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:59.077 7f04ac4a5348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 00:51:59.077 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 00:51:59.077 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 00:51:59.078 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:59.078 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 00:51:59.078 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:59.078 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:59.078 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:59.078 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:52:00.646 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738803120646366, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:52:00.649 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:52:00.650 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = Native-Transport-Requests active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 02/06/2025 00:52:00.650 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:52:03.078 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:52:03.081 7f04ab639348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 00:52:03.081 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:52:03.081 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:52:03.081 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:52:03.081 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:52:03.082 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 00:52:03.082 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 00:52:03.082 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 00:52:03.082 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:52:03.082 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:52:03.086 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 00:52:03.086 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 00:52:03.086 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 00:52:03.087 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 00:52:03.087 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 00:52:03.087 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 00:52:03.087 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:52:03.087 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:52:03.087 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 00:52:03.088 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 00:52:03.089 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 00:52:06.065 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14105436, partition_space_available_1k=66985048, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1325920, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9996, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52104, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7740148, free=449096, buffers=658180, cached=7328212, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.7, five_min_avg=4.49, fifteen_min_avg=2.61, cpu_share=18.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:52:09.905 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 02/06/2025 00:52:09.905 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 00:52:09.906 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/06/2025 00:52:09.906 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 02/06/2025 00:52:09.906 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 00:52:09.906 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 00:52:09.906 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:52:09.906 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:52:09.907 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 00:52:09.907 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 00:52:56.177 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:53:06.032 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14257452, partition_space_available_1k=66833032, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1323260, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10140, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51796, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8573356, free=410576, buffers=665068, cached=6526636, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.23, five_min_avg=4.95, fifteen_min_avg=2.89, cpu_share=19.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:53:56.162 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:54:06.037 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14319448, partition_space_available_1k=66771036, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1329916, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10168, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52236, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8496972, free=434024, buffers=665600, cached=6579040, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.08, five_min_avg=4.87, fifteen_min_avg=3.0, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:54:35.323 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:54:56.172 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:55:06.052 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14777836, partition_space_available_1k=66312648, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364404, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10336, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52236, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8541012, free=368628, buffers=670872, cached=6595124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.48, five_min_avg=5.27, fifteen_min_avg=3.25, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:55:13.937 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:55:56.161 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:56:06.059 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14784988, partition_space_available_1k=66305496, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341732, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10356, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8541852, free=375612, buffers=670732, cached=6587440, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.59, five_min_avg=5.28, fifteen_min_avg=3.39, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:56:56.218 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:57:06.065 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14798872, partition_space_available_1k=66291612, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347396, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10432, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8577276, free=350360, buffers=670908, cached=6577092, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.34, five_min_avg=4.99, fifteen_min_avg=3.4, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:57:56.192 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:58:06.076 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14794292, partition_space_available_1k=66296192, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354488, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10324, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8582720, free=357552, buffers=670656, cached=6564708, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.27, five_min_avg=4.85, fifteen_min_avg=3.46, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:58:56.191 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:59:06.074 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14808300, partition_space_available_1k=66282184, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355372, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10356, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8616084, free=352560, buffers=670456, cached=6536536, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=4.62, fifteen_min_avg=3.47, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:59:56.203 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:00:06.096 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14818436, partition_space_available_1k=66272048, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1362368, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10364, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8674168, free=358568, buffers=670464, cached=6472436, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=4.38, fifteen_min_avg=3.45, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:00:56.329 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:01:06.104 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14826580, partition_space_available_1k=66263904, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1368092, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10372, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8723332, free=329112, buffers=670500, cached=6452692, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.15, five_min_avg=4.13, fifteen_min_avg=3.43, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:01:56.200 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:02:05.108 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14841104, partition_space_available_1k=66249380, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1372840, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10428, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8722348, free=352960, buffers=670524, cached=6429804, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=4.1, fifteen_min_avg=3.46, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:02:56.232 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:03:06.096 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14848612, partition_space_available_1k=66241872, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377136, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10452, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52244, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8758584, free=333812, buffers=670552, cached=6412688, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.95, five_min_avg=4.11, fifteen_min_avg=3.51, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:03:56.268 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:04:06.098 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14859700, partition_space_available_1k=66230784, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1381724, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10460, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52256, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8780404, free=354000, buffers=670596, cached=6370636, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.26, five_min_avg=4.15, fifteen_min_avg=3.56, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:04:56.285 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:05:06.115 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14866888, partition_space_available_1k=66223596, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1386820, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10536, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52256, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8788988, free=345912, buffers=670600, cached=6370136, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.35, five_min_avg=4.17, fifteen_min_avg=3.6, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:05:56.209 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:06:06.116 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14883188, partition_space_available_1k=66207296, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1390784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10616, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52252, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8779924, free=408404, buffers=670572, cached=6316736, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.7, five_min_avg=3.98, fifteen_min_avg=3.57, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:06:56.241 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:07:06.136 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14887180, partition_space_available_1k=66203304, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1395896, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10636, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52272, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8807568, free=375508, buffers=670580, cached=6321980, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.99, five_min_avg=3.71, fifteen_min_avg=3.5, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:07:56.388 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:08:06.142 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14900080, partition_space_available_1k=66190404, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1399504, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8827384, free=352656, buffers=670484, cached=6325112, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=3.88, fifteen_min_avg=3.57, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:08:56.221 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:09:06.118 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14911556, partition_space_available_1k=66178928, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1404336, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10780, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8830104, free=364824, buffers=670380, cached=6310328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.77, five_min_avg=3.8, fifteen_min_avg=3.56, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:09:56.224 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:10:07.121 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14922188, partition_space_available_1k=66168296, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1410352, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10648, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8829384, free=410636, buffers=670340, cached=6265276, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=3.7, fifteen_min_avg=3.54, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:10:56.239 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:11:06.149 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14932172, partition_space_available_1k=66158312, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1413488, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10688, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52292, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8875572, free=362784, buffers=670348, cached=6266932, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.61, five_min_avg=3.67, fifteen_min_avg=3.54, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:11:56.295 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:12:06.167 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14940464, partition_space_available_1k=66150020, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1419444, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10688, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52300, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8869008, free=362828, buffers=670344, cached=6273456, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.29, five_min_avg=3.91, fifteen_min_avg=3.63, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:12:56.290 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:13:06.170 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14935816, partition_space_available_1k=66154668, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1425360, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10732, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52304, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8874564, free=362732, buffers=670364, cached=6267976, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.36, five_min_avg=3.73, fifteen_min_avg=3.59, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:13:56.295 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:14:07.178 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14944872, partition_space_available_1k=66145612, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1429728, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10752, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52300, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8883720, free=393568, buffers=670332, cached=6228016, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=3.62, fifteen_min_avg=3.56, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:14:56.295 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:15:05.182 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14956312, partition_space_available_1k=66134172, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1434088, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10764, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8882892, free=395072, buffers=670328, cached=6227344, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.21, five_min_avg=3.53, fifteen_min_avg=3.53, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:15:56.307 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:16:06.196 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14959952, partition_space_available_1k=66130532, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1437460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10664, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52304, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8899128, free=368296, buffers=670328, cached=6237884, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.46, fifteen_min_avg=3.51, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:16:56.328 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:17:06.220 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14964040, partition_space_available_1k=66126444, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1443660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10684, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8894080, free=370864, buffers=670328, cached=6240364, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.43, fifteen_min_avg=3.49, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:17:56.303 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:18:07.210 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14981184, partition_space_available_1k=66109300, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1449388, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10708, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8883192, free=411348, buffers=670296, cached=6210800, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.39, fifteen_min_avg=3.48, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:18:56.325 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:19:06.222 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14984796, partition_space_available_1k=66105688, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1454204, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10768, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8910000, free=383216, buffers=670292, cached=6212128, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=3.48, fifteen_min_avg=3.5, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:19:56.309 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:20:06.219 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14992052, partition_space_available_1k=66098432, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1458916, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10832, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52312, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8909024, free=381212, buffers=670296, cached=6215104, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=3.39, fifteen_min_avg=3.47, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:20:56.319 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:21:06.250 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14999428, partition_space_available_1k=66091056, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1463756, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10852, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52312, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8896920, free=392716, buffers=670296, cached=6215704, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.47, fifteen_min_avg=3.49, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:21:56.327 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:22:07.243 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15013004, partition_space_available_1k=66077480, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1468192, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10752, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8916976, free=396308, buffers=670276, cached=6192076, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.37, fifteen_min_avg=3.45, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:22:56.331 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:23:06.243 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15017508, partition_space_available_1k=66072976, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1471964, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10768, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52312, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8922056, free=402000, buffers=670264, cached=6181316, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.64, five_min_avg=3.17, fifteen_min_avg=3.37, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:23:56.312 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:24:06.232 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15029432, partition_space_available_1k=66061052, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1476876, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10824, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52316, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8944136, free=373160, buffers=670268, cached=6188072, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.15, five_min_avg=3.23, fifteen_min_avg=3.38, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:24:56.324 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:25:06.265 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15031860, partition_space_available_1k=66058624, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1481184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52316, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8940408, free=369276, buffers=670272, cached=6195680, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.27, five_min_avg=3.27, fifteen_min_avg=3.38, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:25:56.364 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:26:07.235 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15040312, partition_space_available_1k=66050172, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1485268, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8916792, free=413532, buffers=670256, cached=6175056, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.22, fifteen_min_avg=3.36, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:26:56.333 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:27:05.247 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15045440, partition_space_available_1k=66045044, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1491524, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10860, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52312, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8940196, free=381636, buffers=670256, cached=6183548, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.13, five_min_avg=3.2, fifteen_min_avg=3.34, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:27:56.346 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:28:05.241 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15055016, partition_space_available_1k=66035468, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1496000, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10872, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52320, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8935172, free=377956, buffers=670256, cached=6192252, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=3.27, fifteen_min_avg=3.36, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:28:56.373 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:29:06.254 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15062000, partition_space_available_1k=66028484, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1501044, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10812, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52324, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8952796, free=355188, buffers=670260, cached=6197392, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.23, fifteen_min_avg=3.34, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:29:56.383 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:30:07.284 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15071728, partition_space_available_1k=66018756, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1504772, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10828, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52280, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8941776, free=427144, buffers=670176, cached=6136540, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.89, five_min_avg=3.37, fifteen_min_avg=3.38, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:30:56.459 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:31:06.280 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15077388, partition_space_available_1k=66013096, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1510800, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10844, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8960180, free=403360, buffers=670176, cached=6141920, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.33, five_min_avg=3.31, fifteen_min_avg=3.36, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:31:56.380 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:32:05.305 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15088280, partition_space_available_1k=66002204, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1515220, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10860, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52292, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8967712, free=391880, buffers=670180, cached=6145864, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.32, fifteen_min_avg=3.36, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:32:56.416 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:33:06.295 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15108684, partition_space_available_1k=65981800, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1501300, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10872, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8934920, free=397064, buffers=670180, cached=6173472, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.3, fifteen_min_avg=3.35, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:33:56.411 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:34:07.318 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15112328, partition_space_available_1k=65978156, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1507956, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10840, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52292, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8928236, free=410192, buffers=670180, cached=6167028, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=3.3, fifteen_min_avg=3.34, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:34:56.411 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:35:06.321 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15118180, partition_space_available_1k=65972304, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1502904, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10864, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52292, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8941620, free=386116, buffers=670180, cached=6177720, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.95, five_min_avg=3.18, fifteen_min_avg=3.3, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:35:56.456 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:36:06.323 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15129216, partition_space_available_1k=65961268, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1508048, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10924, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8943800, free=382668, buffers=670180, cached=6178988, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.08, five_min_avg=3.19, fifteen_min_avg=3.3, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:36:56.436 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:37:06.337 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15137140, partition_space_available_1k=65953344, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1514308, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10924, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52300, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8944168, free=376176, buffers=670176, cached=6185116, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.33, fifteen_min_avg=3.33, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:37:56.444 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:38:07.332 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15146968, partition_space_available_1k=65943516, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1517068, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10948, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8937312, free=400692, buffers=670152, cached=6167480, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=3.39, fifteen_min_avg=3.36, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:38:56.437 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:39:06.340 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15150196, partition_space_available_1k=65940288, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1520232, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10976, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52300, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8954952, free=378064, buffers=670156, cached=6172464, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.32, five_min_avg=3.35, fifteen_min_avg=3.34, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:39:56.438 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:40:06.352 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15161788, partition_space_available_1k=65928696, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1525504, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10872, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52304, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8949888, free=380976, buffers=670156, cached=6174616, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=3.35, fifteen_min_avg=3.34, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:40:56.416 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:41:05.345 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15167052, partition_space_available_1k=65923432, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1529828, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10924, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8957200, free=362180, buffers=670156, cached=6186100, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.94, five_min_avg=3.24, fifteen_min_avg=3.31, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:41:56.423 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:42:07.359 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15177376, partition_space_available_1k=65913108, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1535116, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10940, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52304, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8943980, free=397608, buffers=670140, cached=6163908, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=3.26, fifteen_min_avg=3.31, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:42:56.421 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:43:05.355 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15184740, partition_space_available_1k=65905744, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1535804, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8962740, free=407928, buffers=670100, cached=6134868, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=3.4, fifteen_min_avg=3.35, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:43:56.456 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:44:06.360 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15192512, partition_space_available_1k=65897972, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1539568, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10972, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52292, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8978816, free=383112, buffers=670100, cached=6143608, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.48, five_min_avg=3.38, fifteen_min_avg=3.35, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:44:56.456 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:45:06.357 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15205172, partition_space_available_1k=65885312, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1543116, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10976, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52304, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8983596, free=369996, buffers=670100, cached=6151944, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.4, fifteen_min_avg=3.36, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:45:56.483 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:46:07.360 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15205380, partition_space_available_1k=65885104, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1548096, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52308, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8961308, free=414228, buffers=670064, cached=6130036, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.65, five_min_avg=3.44, fifteen_min_avg=3.38, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:46:56.476 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:47:06.371 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15216592, partition_space_available_1k=65873892, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1552504, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10916, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52320, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8970532, free=404896, buffers=670060, cached=6130148, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.06, five_min_avg=3.31, fifteen_min_avg=3.33, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:47:56.566 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:48:05.384 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15225332, partition_space_available_1k=65865152, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1558108, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10916, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52320, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8967348, free=400356, buffers=670080, cached=6137852, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.91, five_min_avg=3.47, fifteen_min_avg=3.39, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:48:56.497 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:49:06.392 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15229688, partition_space_available_1k=65860796, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1561408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10916, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52320, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8952016, free=400896, buffers=670088, cached=6152636, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.43, five_min_avg=3.39, fifteen_min_avg=3.36, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:49:56.561 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:50:08.458 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15572192, partition_space_available_1k=65518292, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1548872, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10932, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52108, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9154952, free=363668, buffers=678608, cached=5978408, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.06, five_min_avg=3.78, fifteen_min_avg=3.49, cpu_share=17.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:50:49.749 7f04ac4a5348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:50:49.751 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/06/2025 01:50:49.757 7f04ab639048 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:50:49.758 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/06/2025 01:50:49.758 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:50:49.758 7f04ab639048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/06/2025 01:50:49.759 7f04ab639048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:50:49.759 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:50:49.760 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:50:49.760 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:50:49.760 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:50:49.761 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:50:49.809 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 01:50:49.810 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:50:49.810 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:50:49.811 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:50:49.811 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:50:49.811 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:50:49.812 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:50:49.812 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:50:49.812 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:50:49.813 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:50:49.814 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:50:56.528 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:51:07.418 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15605720, partition_space_available_1k=65484764, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1552808, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9233652, free=376116, buffers=679780, cached=5886088, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.73, five_min_avg=3.9, fifteen_min_avg=3.55, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:51:56.892 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:52:11.535 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16925620, partition_space_available_1k=64164864, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1527540, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11516, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51472, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9437496, free=372540, buffers=725864, cached=5639736, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.3, five_min_avg=5.52, fifteen_min_avg=4.12, cpu_share=22.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:52:56.902 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:53:04.703 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738806784703502, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=1)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:53:10.442 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17977912, partition_space_available_1k=63112572, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1461940, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11516, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51120, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9386396, free=382900, buffers=731188, cached=5675152, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=11.04, five_min_avg=6.66, fifteen_min_avg=4.61, cpu_share=20.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:53:29.969 7f04ac4a5348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:29.972 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/06/2025 01:53:29.981 7f04ab639048 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:29.983 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/06/2025 01:53:29.985 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:29.985 7f04ab639048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:29.989 7f04ab639048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:29.989 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:29.990 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:29.996 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:29.996 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:29.997 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.133 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 01:53:30.134 7f04ac4a5348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:53:30.135 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:53:30.135 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:53:30.135 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:53:30.136 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:53:30.136 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:53:30.137 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:53:30.137 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:53:30.138 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:53:30.140 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:53:30.143 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:30.143 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Error sending message 02/06/2025 01:53:30.144 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/06/2025 01:53:30.149 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:30.149 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/06/2025 01:53:30.150 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:30.151 7f04ac4a5448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:30.151 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:30.152 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:30.153 7f04ac4a5448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:30.154 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:30.155 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.156 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.257 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:30.258 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:30.258 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:30.258 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.259 7f04ac4a5448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:30.259 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.259 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.259 7f04ac4a5448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:30.259 7f04ac4a5448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:34.261 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:34.262 7f04ab639348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:34.262 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:34.262 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:34.262 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:34.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:34.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:34.263 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:34.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:34.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:34.263 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:42.799 7f04ac4a5448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:42.799 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:42.800 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.801 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:46.805 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:46.805 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:46.805 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:46.805 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.806 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.807 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:46.807 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.805 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:50.807 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:50.807 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:50.807 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:50.807 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:50.807 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:50.807 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:50.808 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.809 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:54.810 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:54.810 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:54.810 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:54.810 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:54.810 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:54.810 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:54.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:56.455 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:53:56.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:56.811 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 17932848 disk_space_available_1k = 63157636 analytics_db_size_1k = 24756 >>, ] >> 02/06/2025 01:53:56.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.811 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:58.812 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:58.813 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:58.813 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:58.813 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:58.813 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:58.813 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:58.814 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:59.806 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738806839806353, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=2)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:53:59.809 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:59.810 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = Native-Transport-Requests active = 0 pending = 0 all_time_blocked = 2 >>, ] >> 02/06/2025 01:53:59.810 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.813 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:02.815 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:02.815 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:02.815 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:02.815 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:02.815 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:02.815 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:02.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:05.639 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17933224, partition_space_available_1k=63157260, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1465852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12168, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51180, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7535960, free=2291232, buffers=731956, cached=5616488, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.02, five_min_avg=6.04, fifteen_min_avg=4.52, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:54:05.640 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:05.640 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17933224 partition_space_available_1k = 63157260 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1465852 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12168 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51180 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7535960 free = 2291232 buffers = 731956 cached = 5616488 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.02 five_min_avg = 6.04 fifteen_min_avg = 4.52 cpu_share = 11.0 node_type = database-node >> >> 02/06/2025 01:54:05.640 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.815 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:06.816 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:06.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:06.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:06.816 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:06.816 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:06.817 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.820 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:10.821 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:10.821 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:10.821 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:10.821 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.822 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:10.823 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.825 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:14.826 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:14.826 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:14.826 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.827 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.828 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:14.828 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.829 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:18.830 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:18.831 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:18.831 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:18.831 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:18.831 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:18.831 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:18.832 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:18.832 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.833 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:18.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.833 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:18.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.831 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:22.832 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:22.832 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:22.832 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:22.832 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.833 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.834 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:22.834 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.836 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:26.837 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:26.837 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:26.837 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:26.837 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:26.837 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:26.838 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.840 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:30.841 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:30.841 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:30.841 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:30.841 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:30.841 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:30.842 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:30.842 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:30.842 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.842 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:30.842 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.842 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.843 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:30.843 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.844 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:34.845 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:34.845 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:34.845 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:34.845 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:34.845 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.846 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:34.847 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.847 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:38.855 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:38.855 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:38.855 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:38.856 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.857 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.857 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:38.857 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.864 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:42.865 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:42.865 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:42.865 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:42.865 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:42.865 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:42.865 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:42.866 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.868 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:46.870 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:46.871 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.871 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:46.871 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.871 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.871 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:46.871 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.873 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:50.875 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:50.875 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:50.875 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:50.876 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:50.876 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:50.876 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:50.877 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:50.877 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.877 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:50.878 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.878 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.878 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:50.878 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.877 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:54.878 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:54.878 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:54.878 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:54.879 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:54.879 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:54.879 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:54.879 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:54.879 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.879 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:54.880 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.880 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.880 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:54.880 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:56.672 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:54:57.184 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:57.185 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 17957000 disk_space_available_1k = 63133484 analytics_db_size_1k = 24756 >>, ] >> 02/06/2025 01:54:57.185 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.879 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:58.880 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:58.880 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:58.880 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:58.880 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.881 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.882 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:58.882 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.882 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:02.884 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:02.884 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:02.884 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:02.884 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:02.884 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:02.884 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:02.885 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:02.885 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.885 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:02.885 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.885 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.886 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:02.886 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.672 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17958144, partition_space_available_1k=63132340, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1466260, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12332, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51268, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7890276, free=1884328, buffers=735804, cached=5665228, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.18, five_min_avg=5.76, fifteen_min_avg=4.51, cpu_share=16.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:55:06.672 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.673 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17958144 partition_space_available_1k = 63132340 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1466260 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12332 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51268 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7890276 free = 1884328 buffers = 735804 cached = 5665228 node_type = database-node >> system_cpu_usage = << one_min_avg = 5.18 five_min_avg = 5.76 fifteen_min_avg = 4.51 cpu_share = 16.0 node_type = database-node >> >> 02/06/2025 01:55:06.673 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.893 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:06.936 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:06.936 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:06.936 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:06.936 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.937 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.938 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:06.938 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.940 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:10.941 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:10.941 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:10.941 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.942 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.943 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:10.943 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.944 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:14.945 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:14.946 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:14.946 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:14.946 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:14.946 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:14.946 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:14.946 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:14.947 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.947 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:14.947 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.947 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.947 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:14.947 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.948 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:18.949 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:18.949 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:18.949 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:18.949 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:18.949 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:18.949 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:18.950 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.949 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:22.950 7f04ab639348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:22.950 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:22.951 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.952 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.952 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1738803055263208 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.41:8086, 10.0.0.23:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:22.952 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:26.953 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:26.955 7f04ab639348 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:55:26.955 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:26.955 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:26.955 7f04ab639348 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:26.956 7f04ab639348 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:26.956 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:55:26.956 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:55:26.956 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:55:26.957 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:55:26.957 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:55:28.416 7f04ac4a5448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 02/06/2025 01:55:28.417 7f04ac4a5448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:55:28.418 7f04ab639348 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:55:28.418 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:55:28.420 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:55:28.420 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:55:28.420 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:55:28.420 7f04ab639348 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:55:28.421 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:55:28.421 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:55:28.426 7f04ab639048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:55:38.354 7f04ac4a5448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001584 02/06/2025 01:55:38.355 7f04ac4a5448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:55:38.356 7f04ab639048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/06/2025 01:55:38.356 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 02/06/2025 01:55:38.356 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:55:38.356 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:55:38.357 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:55:38.357 7f04ab639048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:55:38.358 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:55:38.359 7f04ab639348 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:55:56.591 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:56:06.387 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18505860, partition_space_available_1k=62584624, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1467044, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12588, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51296, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9050640, free=552576, buffers=738788, cached=5833632, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.02, five_min_avg=6.45, fifteen_min_avg=4.83, cpu_share=20.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:56:56.555 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:57:05.405 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18520068, partition_space_available_1k=62570416, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1475460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12596, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51304, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9055356, free=526304, buffers=740808, cached=5853168, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.75, five_min_avg=5.81, fifteen_min_avg=4.71, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:57:56.528 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:58:05.605 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18827584, partition_space_available_1k=62262900, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1476560, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12612, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9145924, free=305288, buffers=706604, cached=6017820, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.16, five_min_avg=5.38, fifteen_min_avg=4.62, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:58:56.599 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:59:05.619 7f04b1cb3df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19693384, partition_space_available_1k=61397100, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1562992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=0, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51096, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8177772, free=838428, buffers=717420, cached=6442016, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.27, five_min_avg=5.33, fifteen_min_avg=4.65, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:59:26.953 7fea0e33e708 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 01:59:26.953 7fea0e33e708 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 01:59:26.953 7fea0e33e708 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 01:59:26.962 7fea0bb6ba20 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/06/2025 01:59:26.962 7fea0bb6ba20 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/06/2025 01:59:26.963 7fea0bb72390 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 01:59:26.964 7fea0bb72390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738807166964983 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 01:59:26.965 7fea0bb72390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:27.179 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 63589143334532899916675752466890049617765544733541842143276869285878423266680, 'start': '1738807164000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 20969915258032849801605468517258288394267529314911257550357958535959480039297, 'start': '1738807164000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 62488657593089259470700410431407599983089212526233350687077944217093095416242, 'start': '1738807164000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 01:59:27.355 7fea1459ea20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:27.356 7fea1459ea20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 02/06/2025 01:59:27.356 7fea1459ea20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:27.356 7fea0e33e708 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738807167356737, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:59:27.357 7fea0e33e708 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738807167357087, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807164000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807164000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807164000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:59:27.359 7fea0a8812e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:27.359 7fea0a8812e8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:59:27.359 7fea0a8812e8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:27.359 7fea0a8812e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:27.359 7fea0a8812e8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807164000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807164000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807164000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 02/06/2025 01:59:27.359 7fea0a8812e8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:28.025 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:59:28.534 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:59:28.535 7fea0bf07588 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:28.535 7fea0bf07588 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 19559224 disk_space_available_1k = 61531260 analytics_db_size_1k = 41024 >>, ] >> 02/06/2025 01:59:28.536 7fea0bf07588 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:29.262 7fea0e33e708 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 01:59:29.262 7fea0e33e708 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738807169262199, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:59:29.264 7fea0dd84978 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:29.264 7fea0dd84978 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:59:29.264 7fea0dd84978 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:59:30.966 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:59:30.967 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:59:30.967 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:59:30.967 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:59:30.967 7fea0a877e10 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:59:30.967 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:59:30.967 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:59:30.968 7fea0a877e10 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:59:30.968 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:59:30.968 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:59:30.968 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:59:30.973 7fea0bb70198 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 01:59:30.973 7fea0bb70198 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:59:30.974 7fea0a877048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:59:30.974 7fea0a877048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:59:30.974 7fea0a877048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:59:30.974 7fea0a877048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:59:30.974 7fea0a877048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:59:30.975 7fea0a877048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:59:30.975 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:59:30.976 7fea0a877e10 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:59:31.249 7fea0e33e708 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 02/06/2025 01:59:33.656 7fea0e33e708 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 2 02/06/2025 01:59:38.912 7fea0e33e708 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738807178912223, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807164000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807164000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807164000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:59:38.912 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19562720, partition_space_available_1k=61527764, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1292400, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9832, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51432, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8322096, free=702848, buffers=719584, cached=6431108, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.24, five_min_avg=5.34, fifteen_min_avg=4.68, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:00:27.665 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:00:28.786 7fea0e33e708 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738807228786726, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.23:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:31.151 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='Database', _instance_id='0', _timestamp=1738807231151375, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:31.152 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 02:00:36.937 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19567728, partition_space_available_1k=61522756, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1303088, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10136, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51648, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8357580, free=661584, buffers=719608, cached=6436864, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.02, five_min_avg=4.71, fifteen_min_avg=4.5, cpu_share=10.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:01:27.726 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:01:36.925 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19572200, partition_space_available_1k=61518284, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1328232, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10376, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51656, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8466416, free=545856, buffers=719664, cached=6443700, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=4.57, fifteen_min_avg=4.47, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:02:27.816 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:02:37.950 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19577488, partition_space_available_1k=61512996, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354428, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11448, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51736, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8539476, free=463812, buffers=719672, cached=6452676, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=4.15, fifteen_min_avg=4.32, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:03:27.709 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:03:41.014 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19624392, partition_space_available_1k=61466092, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340200, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11504, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51740, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8715840, free=479984, buffers=709240, cached=6270572, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.88, five_min_avg=4.77, fifteen_min_avg=4.53, cpu_share=17.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:04:27.768 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:04:36.945 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19630908, partition_space_available_1k=61459576, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1345424, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11548, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51744, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8653956, free=509108, buffers=710472, cached=6302100, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.34, five_min_avg=4.59, fifteen_min_avg=4.48, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:05:27.773 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:05:36.960 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19637328, partition_space_available_1k=61453156, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352120, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51708, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8632084, free=524964, buffers=718532, cached=6300056, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.39, five_min_avg=4.53, fifteen_min_avg=4.47, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:06:27.783 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:06:37.955 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19637700, partition_space_available_1k=61452784, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1362864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10884, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51716, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8645092, free=502396, buffers=720448, cached=6307700, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=4.26, fifteen_min_avg=4.37, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:07:27.790 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:07:36.954 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19643212, partition_space_available_1k=61447272, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364524, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10964, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51716, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8645396, free=481084, buffers=722824, cached=6326332, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.41, five_min_avg=3.81, fifteen_min_avg=4.21, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:08:27.815 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:08:36.965 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19647068, partition_space_available_1k=61443416, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369988, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10988, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51716, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8694124, free=453128, buffers=721004, cached=6307380, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.71, fifteen_min_avg=4.15, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:09:27.776 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:09:36.978 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19652308, partition_space_available_1k=61438176, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1375900, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10988, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51704, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8525420, free=612212, buffers=719668, cached=6318336, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.27, five_min_avg=3.65, fifteen_min_avg=4.1, cpu_share=17.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:10:27.827 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:10:37.988 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19676476, partition_space_available_1k=61414008, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1394576, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11060, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51644, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8651844, free=510676, buffers=718648, cached=6294468, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.65, fifteen_min_avg=4.07, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:11:27.801 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:11:36.970 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19693452, partition_space_available_1k=61397032, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1389208, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11064, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51656, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8695336, free=461392, buffers=719196, cached=6299712, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.75, five_min_avg=3.42, fifteen_min_avg=3.97, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:12:27.812 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:12:37.991 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19697168, partition_space_available_1k=61393316, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1397436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11100, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51652, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8682992, free=466012, buffers=719520, cached=6307112, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.47, five_min_avg=3.23, fifteen_min_avg=3.86, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:13:27.872 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:13:38.010 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19701624, partition_space_available_1k=61388860, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1407520, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11132, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51660, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8646048, free=495396, buffers=719748, cached=6314444, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=3.28, fifteen_min_avg=3.84, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:14:07.253 7fea0bb24828 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:14:27.836 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:14:31.082 7fea0e10e6d8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:14:37.005 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20057836, partition_space_available_1k=61032648, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1405292, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11240, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51532, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8602648, free=398196, buffers=719448, cached=6455344, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.73, five_min_avg=3.14, fifteen_min_avg=3.76, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:14:46.305 7fea0ba0ecc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:00.609 7fea0f44fc18 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:27.862 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:15:29.556 7fea0dc1e438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:37.062 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20063560, partition_space_available_1k=61026924, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1421408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11312, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51536, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8591212, free=400140, buffers=719476, cached=6464808, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.06, fifteen_min_avg=3.69, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:16:27.837 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:16:37.039 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21121892, partition_space_available_1k=59968592, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1413540, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11320, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50728, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8692876, free=324360, buffers=717528, cached=6440872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.34, five_min_avg=2.86, fifteen_min_avg=3.58, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:16:54.367 7fea0babe978 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:17:27.857 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:17:40.060 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21170512, partition_space_available_1k=59919972, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1428476, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12460, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50564, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8860420, free=341416, buffers=711248, cached=6262552, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=3.02, fifteen_min_avg=3.58, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:18:27.820 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:18:37.051 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21186648, partition_space_available_1k=59903836, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1454812, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12584, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50420, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9071016, free=319828, buffers=705276, cached=6079516, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.74, five_min_avg=3.56, fifteen_min_avg=3.72, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:19:27.867 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:19:38.060 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21202428, partition_space_available_1k=59888056, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1468040, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12692, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50280, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9141584, free=371028, buffers=699340, cached=5963684, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.52, five_min_avg=3.71, fifteen_min_avg=3.77, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:20:28.021 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:20:38.117 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21234664, partition_space_available_1k=59855820, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1486556, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12708, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50260, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9151812, free=353372, buffers=698812, cached=5971640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.66, five_min_avg=4.11, fifteen_min_avg=3.9, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:21:28.033 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:21:38.088 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21246344, partition_space_available_1k=59844140, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1501376, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12768, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50256, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9159268, free=354820, buffers=698336, cached=5963212, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.53, five_min_avg=4.31, fifteen_min_avg=3.98, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:22:27.863 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:22:37.075 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21283468, partition_space_available_1k=59807016, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1527128, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50232, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9146552, free=376472, buffers=695948, cached=5956664, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.28, five_min_avg=4.43, fifteen_min_avg=4.04, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:23:27.942 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:23:32.443 7fea0dca5cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:23:38.093 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21294364, partition_space_available_1k=59796120, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1528076, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50192, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8987188, free=555120, buffers=691912, cached=5941416, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=4.01, fifteen_min_avg=3.92, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:24:27.868 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:24:37.079 7fea0e33e708 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21308588, partition_space_available_1k=59781896, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1538900, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8965140, free=494912, buffers=695256, cached=6020328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.87, five_min_avg=4.09, fifteen_min_avg=3.95, cpu_share=9.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:25:08.608 7fea0bb72588 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map