03/20/2025 00:48:37.229 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/20/2025 00:48:37.229 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/20/2025 00:48:37.229 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/20/2025 00:48:37.310 7f2abcca3548 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/20/2025 00:48:37.311 7f2abcca3548 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/20/2025 00:48:37.311 7f2abcca3d48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/20/2025 00:48:37.312 7f2abcca3d48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 00:48:37.312 7f2abcca3d48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/20/2025 00:48:37.313 7f2abcca3d48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.515 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 9185836902949213785238956201572882221646809368086316102316709954384500114231, 'start': '1742431714000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 61868596539830148033514237622365451020851136616557657672086087673574030991355, 'start': '1742431714000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 65467823184827247357540752672580658527427242390893910784741048353014935992496, 'start': '1742431714000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/20/2025 00:48:37.658 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.659 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-03-17 01:12:14.132392", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8139-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/20/2025 00:48:37.659 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.660 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431717659875, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:48:37.660 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431717660243, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742431714000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742431714000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742431714000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-03-17 01:12:14.132392", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8139-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:48:37.674 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.675 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 00:48:37.675 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.675 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:37.675 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742431714000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742431714000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742431714000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-03-17 01:12:14.132392", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8139-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/20/2025 00:48:37.675 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:38.030 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:48:38.715 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 00:48:38.717 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:38.717 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. database_usage = [ << disk_space_used_1k = 13654932 disk_space_available_1k = 67435552 analytics_db_size_1k = 4 >>, ] >> 03/20/2025 00:48:38.717 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:39.468 7f2ac23fddf0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/20/2025 00:48:39.469 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431719468948, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:48:39.470 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:39.471 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 00:48:39.471 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.313 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:41.322 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:41.323 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:41.323 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:41.323 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:41.323 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:41.323 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:41.323 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:41.324 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.324 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:41.324 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.324 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:41.324 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:41.324 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:43.903 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431723901741, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=1), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:48:43.903 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 00:48:43.906 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:43.908 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. cassandra_compaction_task = << pending_compaction_tasks = 1 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 03/20/2025 00:48:43.908 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.326 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:45.328 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:45.328 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:45.328 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:45.328 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:45.329 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:45.329 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:45.329 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:45.329 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.329 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:45.330 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.330 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:45.330 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:45.330 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.334 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:49.336 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:49.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.338 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:49.338 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.338 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.338 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:49.338 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.796 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431729796594, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742431714000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742431714000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742431714000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-03-17 01:12:14.132392", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8139-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:48:49.797 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13656496, partition_space_available_1k=67433988, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1267660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7508, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7458896, free=288148, buffers=632328, cached=7796264, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.18, five_min_avg=4.16, fifteen_min_avg=2.33, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:48:49.797 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.798 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742431714000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742431714000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742431714000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-03-17 01:12:14.132392", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8139-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/20/2025 00:48:49.798 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.798 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:49.798 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13656496 partition_space_available_1k = 67433988 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1267660 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7508 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51884 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7458896 free = 288148 buffers = 632328 cached = 7796264 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.18 five_min_avg = 4.16 fifteen_min_avg = 2.33 cpu_share = 0 node_type = database-node >> >> 03/20/2025 00:48:49.806 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.337 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:53.338 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:53.339 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:53.339 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:53.339 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:53.339 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:53.340 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:53.340 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:53.340 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.343 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:53.343 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.343 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:53.343 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:53.343 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.340 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:48:57.341 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.342 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:48:57.343 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.343 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:48:57.343 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:48:57.343 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.344 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:01.346 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:01.346 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:01.346 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:01.346 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:01.346 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:01.346 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:01.347 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.350 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:05.351 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:05.351 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:05.351 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:05.351 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:05.351 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:05.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.351 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:09.352 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:09.352 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:09.353 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:09.353 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:09.353 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:09.353 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:09.353 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:09.353 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.354 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:09.354 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.354 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:09.354 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:09.354 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.354 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:13.355 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:13.355 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:13.355 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:13.355 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:13.356 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.358 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:17.359 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:17.359 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:17.359 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:17.359 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:17.359 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:17.359 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:17.360 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.363 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:21.364 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:21.364 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:21.364 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:21.364 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:21.364 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:21.364 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:21.365 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.364 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:25.365 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 00:49:25.365 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.366 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 00:49:25.367 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.367 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:25.367 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/20/2025 00:49:25.367 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 00:49:29.368 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 00:49:29.369 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/20/2025 00:49:29.370 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 00:49:29.370 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:29.370 7f2abcaf0248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 00:49:29.371 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 00:49:29.371 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 00:49:29.371 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/20/2025 00:49:29.371 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 00:49:29.371 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 00:49:29.372 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 00:49:37.957 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:49:38.446 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 00:49:39.368 7f2ac23fddf0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431779368904, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.47:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:49:39.371 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 00:49:40.062 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/20/2025 00:49:40.062 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 00:49:40.063 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/20/2025 00:49:40.064 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 00:49:40.065 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 00:49:42.657 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742431782657735, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 00:49:47.840 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13969744, partition_space_available_1k=67120740, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1329008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9976, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51600, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7714460, free=355336, buffers=640152, cached=7465688, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.62, five_min_avg=4.32, fifteen_min_avg=2.5, cpu_share=18.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:49:50.814 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 03/20/2025 00:49:50.815 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 00:49:50.815 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/20/2025 00:49:50.818 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 03/20/2025 00:49:50.818 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 00:49:50.818 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 00:49:50.818 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 00:49:50.818 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 00:49:50.818 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 00:49:50.819 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 00:50:38.012 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:50:48.818 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14110516, partition_space_available_1k=66979968, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334812, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10032, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51284, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8611728, free=399388, buffers=647064, cached=6517456, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.5, five_min_avg=4.56, fifteen_min_avg=2.7, cpu_share=19.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:51:37.989 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:51:47.808 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14163200, partition_space_available_1k=66927284, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1332088, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10100, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51860, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8524664, free=436888, buffers=647188, cached=6566896, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.16, five_min_avg=4.66, fifteen_min_avg=2.86, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:52:38.192 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:52:38.483 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 00:52:47.837 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14632520, partition_space_available_1k=66457964, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1375184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10336, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51864, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8581100, free=379736, buffers=652780, cached=6562020, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.98, five_min_avg=4.68, fifteen_min_avg=2.98, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:53:03.785 7f2abcaf0248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 00:53:38.107 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:53:47.813 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14646808, partition_space_available_1k=66443676, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344124, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10424, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51868, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8612508, free=367132, buffers=652876, cached=6543120, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.35, five_min_avg=4.55, fifteen_min_avg=3.04, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:54:38.032 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:54:47.820 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14656180, partition_space_available_1k=66434304, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347716, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10464, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51876, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8631616, free=366776, buffers=652916, cached=6524328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.22, five_min_avg=4.46, fifteen_min_avg=3.11, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:55:38.014 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:55:47.829 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14647440, partition_space_available_1k=66443044, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352672, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10320, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51880, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8691980, free=350364, buffers=652720, cached=6480572, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=4.41, fifteen_min_avg=3.17, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:56:38.061 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:56:47.837 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14662908, partition_space_available_1k=66427576, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355468, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10344, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51876, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8750936, free=334620, buffers=652748, cached=6437332, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.62, five_min_avg=4.2, fifteen_min_avg=3.18, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:57:38.033 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:57:48.851 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14668768, partition_space_available_1k=66421716, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1390756, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10416, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51880, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8796964, free=392004, buffers=653056, cached=6333612, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.57, five_min_avg=4.39, fifteen_min_avg=3.31, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:58:38.061 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:58:47.839 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14681776, partition_space_available_1k=66408708, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1367944, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10496, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51888, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8826800, free=361748, buffers=653144, cached=6333944, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.32, five_min_avg=4.84, fifteen_min_avg=3.53, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 00:59:38.100 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 00:59:47.851 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14684456, partition_space_available_1k=66406028, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1375152, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10512, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8866836, free=372896, buffers=653024, cached=6282880, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.09, five_min_avg=5.01, fifteen_min_avg=3.67, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:00:38.108 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:00:47.851 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14697672, partition_space_available_1k=66392812, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376824, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10520, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51908, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8833600, free=397024, buffers=653148, cached=6291864, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.01, five_min_avg=4.9, fifteen_min_avg=3.72, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:01:38.088 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:01:47.849 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14704068, partition_space_available_1k=66386416, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1379912, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10488, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8891784, free=322292, buffers=653200, cached=6308360, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.71, five_min_avg=4.82, fifteen_min_avg=3.76, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:02:38.086 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:02:47.863 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14710912, partition_space_available_1k=66379572, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1384980, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10492, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8852548, free=365880, buffers=653200, cached=6304008, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.86, five_min_avg=4.82, fifteen_min_avg=3.83, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:03:38.072 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:03:47.848 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14724564, partition_space_available_1k=66365920, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1389656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10596, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8854748, free=350224, buffers=653220, cached=6317444, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.74, five_min_avg=4.54, fifteen_min_avg=3.8, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:04:38.065 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:04:48.856 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14735376, partition_space_available_1k=66355108, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1395440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10676, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51920, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8851656, free=405360, buffers=653156, cached=6265464, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.33, five_min_avg=4.5, fifteen_min_avg=3.83, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:05:38.105 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:05:47.853 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14749808, partition_space_available_1k=66340676, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1400288, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10720, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51924, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8892212, free=358016, buffers=653156, cached=6272252, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.72, five_min_avg=4.59, fifteen_min_avg=3.9, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:06:38.155 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:06:47.875 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14754064, partition_space_available_1k=66336420, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1405228, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10728, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52012, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8889940, free=361908, buffers=653144, cached=6270644, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.8, five_min_avg=4.32, fifteen_min_avg=3.85, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:07:38.113 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:07:47.908 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14755700, partition_space_available_1k=66334784, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1412132, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10648, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8888240, free=359756, buffers=653092, cached=6274548, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=4.17, fifteen_min_avg=3.83, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:08:38.097 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:08:47.920 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14770992, partition_space_available_1k=66319492, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1415408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10696, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8873140, free=392176, buffers=653076, cached=6257244, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=4.12, fifteen_min_avg=3.83, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:09:38.103 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:09:47.905 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14758812, partition_space_available_1k=66331672, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1423740, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10744, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8923048, free=364272, buffers=653080, cached=6235236, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.27, five_min_avg=3.92, fifteen_min_avg=3.78, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:10:38.184 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:10:47.922 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14759272, partition_space_available_1k=66331212, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1429220, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10768, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8916568, free=356648, buffers=653080, cached=6249340, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.78, fifteen_min_avg=3.74, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:11:38.187 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:11:48.931 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14769544, partition_space_available_1k=66320940, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1431896, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10788, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8920972, free=389472, buffers=653004, cached=6212188, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.09, five_min_avg=4.18, fifteen_min_avg=3.88, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:12:38.212 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:12:47.937 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14779336, partition_space_available_1k=66311148, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1438424, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10896, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52060, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8967504, free=331608, buffers=653004, cached=6223520, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.85, five_min_avg=4.2, fifteen_min_avg=3.9, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:13:38.185 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:13:47.937 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14782828, partition_space_available_1k=66307656, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1443128, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10820, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8948944, free=373856, buffers=652964, cached=6199872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.77, five_min_avg=4.01, fifteen_min_avg=3.85, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:14:38.160 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:14:47.950 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14792584, partition_space_available_1k=66297900, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1452140, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8942896, free=372732, buffers=652976, cached=6207032, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.93, five_min_avg=4.0, fifteen_min_avg=3.86, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:15:38.155 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:15:48.953 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14793012, partition_space_available_1k=66297472, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1450584, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10908, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52036, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8939116, free=403016, buffers=652916, cached=6180588, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.76, five_min_avg=3.94, fifteen_min_avg=3.85, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:16:38.152 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:16:47.975 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14800872, partition_space_available_1k=66289612, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1452820, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10932, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52040, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8953596, free=379100, buffers=652920, cached=6190020, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.07, five_min_avg=3.94, fifteen_min_avg=3.85, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:17:38.182 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:17:47.976 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14805188, partition_space_available_1k=66285296, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1457336, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10940, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52036, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8964124, free=372600, buffers=652924, cached=6185988, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.75, fifteen_min_avg=3.79, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:18:38.195 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:18:47.961 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14814032, partition_space_available_1k=66276452, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1461232, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10960, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52048, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8961080, free=366636, buffers=652928, cached=6194992, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=3.71, fifteen_min_avg=3.77, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:19:38.193 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:19:47.973 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14816996, partition_space_available_1k=66273488, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1473760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10828, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8936160, free=389260, buffers=652932, cached=6197284, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.36, five_min_avg=3.59, fifteen_min_avg=3.72, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:20:38.186 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:20:47.980 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14825120, partition_space_available_1k=66265364, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1470184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10856, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52048, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8965280, free=361260, buffers=652916, cached=6196180, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.42, five_min_avg=3.86, fifteen_min_avg=3.81, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:21:38.177 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:21:47.973 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14833008, partition_space_available_1k=66257476, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1473444, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10888, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8966704, free=370428, buffers=652892, cached=6185612, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.32, five_min_avg=3.95, fifteen_min_avg=3.84, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:22:38.205 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:22:47.991 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14837976, partition_space_available_1k=66252508, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1479256, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10884, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8983004, free=345440, buffers=652892, cached=6194300, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.01, five_min_avg=3.94, fifteen_min_avg=3.85, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:23:38.162 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:23:47.978 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14843012, partition_space_available_1k=66247472, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1482800, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10888, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9017248, free=313536, buffers=652876, cached=6191976, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.8, five_min_avg=4.12, fifteen_min_avg=3.91, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:24:38.179 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:24:47.981 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14852852, partition_space_available_1k=66237632, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1489044, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10912, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52052, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8979824, free=373324, buffers=652868, cached=6169620, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.58, five_min_avg=4.15, fifteen_min_avg=3.93, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:25:38.240 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:25:47.976 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14858440, partition_space_available_1k=66232044, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1492992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10828, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8970828, free=379060, buffers=652868, cached=6172880, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.55, five_min_avg=4.24, fifteen_min_avg=3.98, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:26:38.306 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:26:48.989 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14866716, partition_space_available_1k=66223768, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1500572, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10856, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8979436, free=400040, buffers=652800, cached=6143360, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.04, five_min_avg=4.13, fifteen_min_avg=3.96, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:27:38.200 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:27:48.011 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14870476, partition_space_available_1k=66220008, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1505340, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8999120, free=378652, buffers=652800, cached=6145064, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.97, five_min_avg=4.08, fifteen_min_avg=3.95, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:28:38.230 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:28:48.017 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14880632, partition_space_available_1k=66209852, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1508512, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9007864, free=377120, buffers=652780, cached=6137872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.55, five_min_avg=4.18, fifteen_min_avg=4.0, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:29:38.202 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:29:48.043 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14887196, partition_space_available_1k=66203288, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1515556, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10976, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8997576, free=375024, buffers=652784, cached=6150252, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.01, five_min_avg=4.09, fifteen_min_avg=3.97, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:30:38.249 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:30:49.053 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14890580, partition_space_available_1k=66199904, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1557244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10988, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52136, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8964632, free=433448, buffers=652740, cached=6124816, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.44, five_min_avg=4.17, fifteen_min_avg=4.01, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:31:38.245 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:31:48.046 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14911856, partition_space_available_1k=66178628, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1495140, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10876, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8973328, free=415188, buffers=652740, cached=6134380, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.4, five_min_avg=4.21, fifteen_min_avg=4.04, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:32:38.245 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:32:48.061 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14912360, partition_space_available_1k=66178124, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1494448, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8968252, free=414772, buffers=652740, cached=6139872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.03, five_min_avg=4.38, fifteen_min_avg=4.1, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:33:38.313 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:33:48.062 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14922916, partition_space_available_1k=66167568, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1499228, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10936, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8972072, free=410280, buffers=652748, cached=6140536, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.09, five_min_avg=4.53, fifteen_min_avg=4.17, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:34:38.279 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:34:48.068 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14928316, partition_space_available_1k=66162168, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1506200, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10952, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8970172, free=402300, buffers=652748, cached=6150416, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.08, five_min_avg=4.57, fifteen_min_avg=4.2, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:35:38.291 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:35:48.080 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14939312, partition_space_available_1k=66151172, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1508784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10952, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8988308, free=377468, buffers=652748, cached=6157112, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.75, five_min_avg=4.61, fifteen_min_avg=4.24, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:36:38.297 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:36:48.078 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14940648, partition_space_available_1k=66149836, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1513308, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10964, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8986756, free=368524, buffers=652748, cached=6167608, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.7, five_min_avg=4.59, fifteen_min_avg=4.26, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:37:38.358 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:37:48.096 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14945360, partition_space_available_1k=66145124, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1520192, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10852, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8983096, free=375552, buffers=652744, cached=6164244, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.59, five_min_avg=4.56, fifteen_min_avg=4.27, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:38:38.273 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:38:48.097 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14954940, partition_space_available_1k=66135544, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1524992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10884, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8963440, free=389860, buffers=652744, cached=6169592, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=4.45, fifteen_min_avg=4.25, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:39:38.360 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:39:48.102 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14961356, partition_space_available_1k=66129128, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1529536, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10908, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52168, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8977124, free=374048, buffers=652744, cached=6171720, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.7, five_min_avg=4.28, fifteen_min_avg=4.2, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:40:38.322 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:40:48.102 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14971560, partition_space_available_1k=66118924, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1536796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10936, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52164, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8983256, free=365840, buffers=652744, cached=6173796, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.69, five_min_avg=4.42, fifteen_min_avg=4.25, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:41:38.353 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:41:49.101 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14977304, partition_space_available_1k=66113180, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1536452, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52144, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8998564, free=377736, buffers=652728, cached=6146608, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.21, five_min_avg=4.54, fifteen_min_avg=4.3, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:42:38.307 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:42:48.115 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14986268, partition_space_available_1k=66104216, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1542036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9000372, free=371360, buffers=652728, cached=6151176, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.85, five_min_avg=4.54, fifteen_min_avg=4.31, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:43:38.306 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:43:48.100 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14985572, partition_space_available_1k=66104912, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1546016, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10880, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52124, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9011536, free=405156, buffers=652676, cached=6106268, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.07, five_min_avg=4.39, fifteen_min_avg=4.28, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:44:38.317 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:44:48.110 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14991920, partition_space_available_1k=66098564, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1549452, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10912, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52128, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9013708, free=400432, buffers=652676, cached=6108820, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.05, five_min_avg=4.34, fifteen_min_avg=4.26, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:45:38.510 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:45:49.103 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15000468, partition_space_available_1k=66090016, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1553760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52132, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8990560, free=415960, buffers=652688, cached=6116428, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.0, five_min_avg=4.25, fifteen_min_avg=4.24, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:46:38.335 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:46:48.118 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15005708, partition_space_available_1k=66084776, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1561572, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52132, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9004852, free=391068, buffers=652688, cached=6127028, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=4.08, fifteen_min_avg=4.18, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:47:38.306 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:47:48.131 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15015396, partition_space_available_1k=66075088, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1565756, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52136, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9007896, free=395064, buffers=652684, cached=6119992, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.33, five_min_avg=3.95, fifteen_min_avg=4.13, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:48:38.341 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:48:48.164 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15019640, partition_space_available_1k=66070844, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1573048, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52136, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9023656, free=370408, buffers=652692, cached=6128880, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=3.84, fifteen_min_avg=4.08, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:49:38.416 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:49:48.150 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15028712, partition_space_available_1k=66061772, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1574864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10872, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9017524, free=411748, buffers=652672, cached=6093692, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.93, five_min_avg=3.97, fifteen_min_avg=4.11, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:50:38.383 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:50:48.161 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15032536, partition_space_available_1k=66057948, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1576436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52148, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9015304, free=404496, buffers=652676, cached=6103160, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.03, five_min_avg=3.95, fifteen_min_avg=4.1, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:51:38.430 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:51:49.201 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15388144, partition_space_available_1k=65702340, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1567448, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10896, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51988, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9204828, free=288388, buffers=658284, cached=6024136, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.3, five_min_avg=3.99, fifteen_min_avg=4.1, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:52:38.446 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:52:46.844 7f2abc9fd248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/20/2025 01:52:46.845 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/20/2025 01:52:46.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/20/2025 01:52:46.849 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/20/2025 01:52:46.849 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:52:46.849 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/20/2025 01:52:46.849 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:52:46.849 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 01:52:46.849 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/20/2025 01:52:46.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 01:52:46.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:52:46.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:52:46.861 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/20/2025 01:52:46.862 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 01:52:46.863 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/20/2025 01:52:46.863 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/20/2025 01:52:46.863 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 01:52:46.863 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 01:52:46.864 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 01:52:46.864 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 01:52:46.864 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/20/2025 01:52:46.865 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 01:52:46.866 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 01:52:47.180 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15408628, partition_space_available_1k=65681856, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1559100, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11640, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9312892, free=427828, buffers=657944, cached=5776972, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.74, five_min_avg=4.23, fifteen_min_avg=4.17, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:53:38.756 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:53:49.046 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742435629046428, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=5)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 01:53:54.240 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16643544, partition_space_available_1k=64446940, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1475436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12464, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51360, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9473596, free=292844, buffers=723360, cached=5685836, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.19, five_min_avg=5.72, fifteen_min_avg=4.68, cpu_share=20.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:54:38.493 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:54:50.201 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17044696, partition_space_available_1k=64045788, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1473180, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12472, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9428284, free=300240, buffers=747064, cached=5700048, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.96, five_min_avg=6.31, fifteen_min_avg=4.96, cpu_share=18.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:55:38.444 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:55:46.008 7f2abc9fd248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/20/2025 01:55:46.008 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/20/2025 01:55:46.017 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/20/2025 01:55:46.017 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/20/2025 01:55:46.017 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:55:46.017 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/20/2025 01:55:46.018 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:55:46.018 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 01:55:46.018 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/20/2025 01:55:46.018 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 01:55:46.018 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:55:46.018 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:55:46.185 7f2abc9fd248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/20/2025 01:55:46.185 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/20/2025 01:55:46.185 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/20/2025 01:55:46.186 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:46.186 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/20/2025 01:55:46.186 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:46.186 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:46.186 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:55:46.187 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:50.187 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17854808, partition_space_available_1k=63235676, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1462328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12160, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50940, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8454364, free=1307768, buffers=729164, cached=5684340, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.32, five_min_avg=6.04, fifteen_min_avg=4.95, cpu_share=18.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:55:50.187 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17854808 partition_space_available_1k = 63235676 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1462328 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12160 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 50940 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 8454364 free = 1307768 buffers = 729164 cached = 5684340 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.32 five_min_avg = 6.04 fifteen_min_avg = 4.95 cpu_share = 18.0 node_type = database-node >> >> 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:55:50.189 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 01:55:50.190 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/20/2025 01:55:50.190 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 01:55:50.190 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:55:50.190 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:55:58.779 7f2abc9fd248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/20/2025 01:55:58.780 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/20/2025 01:55:58.780 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/20/2025 01:55:58.780 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:58.780 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/20/2025 01:55:58.780 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:58.781 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:55:58.781 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:55:58.781 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:02.781 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:02.782 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:02.782 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:02.782 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:02.782 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:02.782 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:02.782 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:02.783 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:06.783 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:06.784 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:06.784 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:06.784 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:06.785 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:06.786 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:06.786 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:10.787 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:10.788 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:10.788 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:10.789 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:10.790 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:10.790 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:10.790 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:14.792 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:14.793 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:14.793 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:14.793 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:14.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:14.795 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:14.795 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:18.794 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:18.795 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:18.795 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:18.795 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:18.795 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:18.795 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:18.795 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:18.796 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:22.798 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:22.799 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:22.799 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:22.799 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:22.799 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:22.799 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:22.799 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:22.800 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:26.802 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:26.804 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:26.804 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:26.804 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:26.804 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:26.804 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:26.805 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:26.805 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:26.805 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:26.805 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:26.805 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:26.806 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:26.806 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:26.806 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:30.806 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:30.807 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:30.807 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:30.807 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:30.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:30.809 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:30.809 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:34.808 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:34.816 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:34.817 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:34.817 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:34.817 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:34.817 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:34.817 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:34.818 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:38.347 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:56:38.680 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:38.680 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. database_usage = [ << disk_space_used_1k = 17751632 disk_space_available_1k = 63338852 analytics_db_size_1k = 24176 >>, ] >> 03/20/2025 01:56:38.680 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:38.817 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:38.819 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:38.820 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:38.820 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:38.820 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:38.820 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:38.820 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:38.820 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:42.820 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:42.823 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:42.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:42.825 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:42.825 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:42.825 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:42.825 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:42.825 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:46.824 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:46.826 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:46.827 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:46.827 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:46.827 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:46.827 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:46.827 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:46.827 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:47.172 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17751456, partition_space_available_1k=63339028, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1459444, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12616, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51024, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7405292, free=2398736, buffers=729428, cached=5642180, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=5.43, fifteen_min_avg=4.8, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:56:47.172 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:47.173 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17751456 partition_space_available_1k = 63339028 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1459444 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12616 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51024 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7405292 free = 2398736 buffers = 729428 cached = 5642180 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.13 five_min_avg = 5.43 fifteen_min_avg = 4.8 cpu_share = 7.75 node_type = database-node >> >> 03/20/2025 01:56:47.173 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:50.827 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:50.828 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:50.828 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:50.828 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:50.828 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:50.828 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:50.828 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:50.829 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:54.830 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:54.832 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:54.832 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:54.832 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:54.833 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:54.834 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:54.834 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:54.834 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:58.834 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:56:58.835 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:56:58.835 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:56:58.835 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:56:58.836 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:56:58.837 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:02.839 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:02.841 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:02.841 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:02.841 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:02.841 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:02.841 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:02.841 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:02.842 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:06.843 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:06.844 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:06.845 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:06.845 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:06.845 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:06.845 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:06.846 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:06.846 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:06.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:06.847 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:06.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:06.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:06.849 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:06.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:10.846 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:10.847 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:10.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:10.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:10.847 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:10.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:10.847 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:10.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:14.848 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:14.849 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:14.849 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:14.849 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:14.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:14.851 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:14.851 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:14.851 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:18.850 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:18.851 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:18.852 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:18.852 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:18.853 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:18.853 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:18.853 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:18.854 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:18.854 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:18.855 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:18.855 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:18.855 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:18.856 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:18.856 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:22.855 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:22.859 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:22.859 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:22.859 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:22.859 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:22.859 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:22.859 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:22.860 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:26.862 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:26.863 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:26.863 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:26.863 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:26.863 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:26.863 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:26.863 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:26.864 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:30.866 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:30.868 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:30.868 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:30.868 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:30.868 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:30.868 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:30.869 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:30.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:34.868 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:34.870 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:34.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:34.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:34.870 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:34.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:34.870 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.47:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:34.871 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:38.430 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:57:38.841 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:38.841 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. database_usage = [ << disk_space_used_1k = 17786324 disk_space_available_1k = 63304160 analytics_db_size_1k = 24176 >>, ] >> 03/20/2025 01:57:38.846 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:38.988 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:38.989 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:38.990 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:38.990 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:38.990 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:38.990 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:38.990 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.42:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.42:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:38.991 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:42.990 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:42.992 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/20/2025 01:57:42.992 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:42.992 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:42.992 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:42.992 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:42.992 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/20/2025 01:57:42.993 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/20/2025 01:57:42.993 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:42.993 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.12:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/20/2025 01:57:42.993 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:42.993 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:42.994 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1742431717312783 collector_name = collector_ip = 10.0.0.12:8086 collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/20/2025 01:57:42.994 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 01:57:46.993 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 01:57:46.995 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/20/2025 01:57:46.995 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 01:57:46.995 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:46.995 7f2abcca3e48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 01:57:46.995 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 01:57:46.995 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 01:57:46.996 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/20/2025 01:57:46.996 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 01:57:46.996 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:57:46.996 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 01:57:47.727 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/20/2025 01:57:47.728 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 01:57:47.728 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/20/2025 01:57:47.728 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/20/2025 01:57:47.729 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 01:57:47.729 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 01:57:47.729 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 01:57:47.729 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 01:57:47.729 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/20/2025 01:57:47.730 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 01:57:47.731 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 01:57:48.200 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17786400, partition_space_available_1k=63304084, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1461924, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12888, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51120, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7952248, free=1772988, buffers=734680, cached=5715720, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.98, five_min_avg=5.46, fifteen_min_avg=4.86, cpu_share=18.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:57:58.296 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001584 03/20/2025 01:57:58.296 7f2abc9fd248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 01:57:58.297 7f2abcca3e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/20/2025 01:57:58.297 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 03/20/2025 01:57:58.297 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 01:57:58.297 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 01:57:58.297 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 01:57:58.297 7f2abcca3e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 01:57:58.297 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 01:57:58.298 7f2abcaf0248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 01:58:38.401 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:58:47.923 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18324340, partition_space_available_1k=62766144, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1462324, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12992, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51116, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9022980, free=471812, buffers=746548, cached=5934296, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.44, five_min_avg=5.54, fifteen_min_avg=4.92, cpu_share=17.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 01:59:38.417 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 01:59:46.642 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18347128, partition_space_available_1k=62743356, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1469940, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13004, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51124, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9051872, free=436600, buffers=746556, cached=5940608, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=4.93, fifteen_min_avg=4.75, cpu_share=9.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:00:38.408 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:00:47.623 7f2ac23fddf0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19155248, partition_space_available_1k=61935236, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1461332, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13004, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50908, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6617736, free=2858936, buffers=709584, cached=5989380, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.46, five_min_avg=4.6, fifteen_min_avg=4.64, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:01:53.556 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/20/2025 02:01:53.556 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/20/2025 02:01:53.556 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/20/2025 02:01:53.565 7f1b7e4f36d8 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/20/2025 02:01:53.567 7f1b7e4f36d8 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/20/2025 02:01:53.568 7f1b7e4fc048 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/20/2025 02:01:53.569 7f1b7e4fc048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 02:01:53.570 7f1b7e4fc048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 02:01:53.570 7f1b7e4fc048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/20/2025 02:01:53.570 7f1b7e4fc048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.571 7f1b7e4fc048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 02:01:53.571 7f1b7e4fc048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.571 7f1b7e4fc048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.571 7f1b7e4fc048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1742436113570329 collector_name = collector_ip = collector_list = [ 10.0.0.47:8086, 10.0.0.42:8086, 10.0.0.12:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/20/2025 02:01:53.571 7f1b7e4fc048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.786 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 35654009842534353346591134609801641538791358537050402429656973535832505981279, 'start': '1742436111000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 103726044130139106931580841237546860124934286685683939882718595835289003762878, 'start': '1742436111000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 75146285275627679467684994764468908851903539871871471733850613843837433486954, 'start': '1742436111000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/20/2025 02:01:53.922 7f1b84a2aa20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.922 7f1b84a2aa20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-20 00:47:32.457292", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8220-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 03/20/2025 02:01:53.922 7f1b84a2aa20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.923 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436113923148, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:01:53.923 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436113923466, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742436111000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742436111000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742436111000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-20 00:47:32.457292", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8220-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:01:53.925 7f1b7c0880f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.925 7f1b7c0880f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 02:01:53.925 7f1b7c0880f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.925 7f1b7c0880f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:53.925 7f1b7c0880f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742436111000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742436111000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1742436111000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-20 00:47:32.457292", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8220-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/20/2025 02:01:53.925 7f1b7c0880f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:54.452 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:01:54.913 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 02:01:54.915 7f1b7beab2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:54.915 7f1b7beab2e8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. database_usage = [ << disk_space_used_1k = 19382320 disk_space_available_1k = 61708164 analytics_db_size_1k = 41344 >>, ] >> 03/20/2025 02:01:54.915 7f1b7beab2e8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:55.464 7f1b7e78ef08 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/20/2025 02:01:55.465 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436115465106, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:01:55.466 7f1b7bebf588 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:55.466 7f1b7bebf588 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/20/2025 02:01:55.466 7f1b7bebf588 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/20/2025 02:01:57.301 7f1b7e78ef08 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 03/20/2025 02:01:57.570 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/20/2025 02:01:57.572 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/20/2025 02:01:57.572 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/20/2025 02:01:57.572 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/20/2025 02:01:57.572 7f1b7c0fc2e8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2923-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.47:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/20/2025 02:01:57.572 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/20/2025 02:01:57.572 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/20/2025 02:01:57.573 7f1b7c0fc2e8 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/20/2025 02:01:57.573 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/20/2025 02:01:57.573 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 02:01:57.574 7f1b7c0fc2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/20/2025 02:01:57.578 7f1b7bfc2b70 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/20/2025 02:01:57.578 7f1b7bfc2b70 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/20/2025 02:01:57.579 7f1b7e514240 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/20/2025 02:01:57.579 7f1b7e514240 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/20/2025 02:01:57.579 7f1b7e514240 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/20/2025 02:01:57.579 7f1b7e514240 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 02:01:57.579 7f1b7e514240 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/20/2025 02:01:57.580 7f1b7e514240 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/20/2025 02:01:57.580 7f1b7bfc2630 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/20/2025 02:01:57.581 7f1b7bfc2630 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/20/2025 02:02:02.687 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436122686943, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742436111000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742436111000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1742436111000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-20 00:47:32.457292", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8220-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:02:02.687 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19377968, partition_space_available_1k=61712516, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1222920, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9760, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51428, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8485308, free=543100, buffers=725188, cached=6422040, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.86, five_min_avg=4.83, fifteen_min_avg=4.72, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:02:54.227 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:02:55.393 7f1b7e78ef08 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436175393838, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.47:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:02:57.821 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436177821309, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:02:57.822 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/20/2025 02:03:03.685 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19386896, partition_space_available_1k=61703588, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1302328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9960, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51604, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8571816, free=448064, buffers=725284, cached=6430472, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.14, five_min_avg=4.63, fifteen_min_avg=4.66, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:03:54.312 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:04:04.730 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19393220, partition_space_available_1k=61697264, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333232, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10264, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51668, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8665904, free=402716, buffers=721796, cached=6385220, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.64, five_min_avg=4.43, fifteen_min_avg=4.59, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:04:54.344 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:05:04.732 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19398156, partition_space_available_1k=61692328, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334364, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11120, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51624, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8693684, free=436168, buffers=719436, cached=6326348, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=4.01, fifteen_min_avg=4.43, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:05:54.341 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:06:03.735 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19436280, partition_space_available_1k=61654204, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339160, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11332, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51600, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8813316, free=334952, buffers=716976, cached=6310392, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=3.7, fifteen_min_avg=4.3, cpu_share=15.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:06:54.305 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:07:03.734 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19439884, partition_space_available_1k=61650600, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344928, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11408, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51524, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8816780, free=407012, buffers=712600, cached=6239244, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.64, five_min_avg=3.51, fifteen_min_avg=4.19, cpu_share=17.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:07:54.315 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:08:03.751 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19445592, partition_space_available_1k=61644892, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10680, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51500, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8828700, free=377280, buffers=721824, cached=6247832, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.74, five_min_avg=3.39, fifteen_min_avg=4.1, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:08:54.464 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:09:04.750 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19451644, partition_space_available_1k=61638840, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10716, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51416, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8862776, free=445704, buffers=718844, cached=6148312, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.88, five_min_avg=3.29, fifteen_min_avg=4.02, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:09:54.333 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:10:03.742 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19457244, partition_space_available_1k=61633240, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1362756, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10732, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51416, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8843444, free=434960, buffers=722308, cached=6174924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.09, five_min_avg=3.29, fifteen_min_avg=3.98, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:10:54.386 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:11:03.752 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19457736, partition_space_available_1k=61632748, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369644, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10808, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51416, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8810444, free=470200, buffers=722332, cached=6172660, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.1, five_min_avg=3.48, fifteen_min_avg=4.0, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:11:54.620 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:12:00.276 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436720275995, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=6, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:12:05.812 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19458404, partition_space_available_1k=61632080, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377772, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10824, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51424, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8846472, free=430852, buffers=722340, cached=6175972, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.96, five_min_avg=3.56, fifteen_min_avg=3.99, cpu_share=16.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:12:54.446 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:13:00.006 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742436780006836, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:13:05.821 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19481508, partition_space_available_1k=61608976, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1385436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10824, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51340, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8853216, free=497680, buffers=717628, cached=6107112, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.17, five_min_avg=4.15, fifteen_min_avg=4.16, cpu_share=17.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:13:54.343 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:14:03.759 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19503984, partition_space_available_1k=61586500, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1396348, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10748, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51336, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8877612, free=423664, buffers=720436, cached=6153924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.1, five_min_avg=3.91, fifteen_min_avg=4.07, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:14:54.363 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:15:03.783 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19502196, partition_space_available_1k=61588288, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1397184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51332, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8826668, free=487924, buffers=720168, cached=6140876, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.97, five_min_avg=3.63, fifteen_min_avg=3.97, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:15:54.347 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:16:03.761 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19507828, partition_space_available_1k=61582656, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1404308, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10904, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51336, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8837028, free=469192, buffers=720324, cached=6149092, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.86, five_min_avg=3.5, fifteen_min_avg=3.9, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:16:54.458 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:17:04.778 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19512500, partition_space_available_1k=61577984, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1403640, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11032, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51336, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8775396, free=526488, buffers=720424, cached=6153328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.6, five_min_avg=3.36, fifteen_min_avg=3.83, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:17:54.362 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:18:03.804 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19515292, partition_space_available_1k=61575192, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1409980, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11132, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51340, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8776920, free=517292, buffers=720428, cached=6160996, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.92, five_min_avg=3.02, fifteen_min_avg=3.68, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:18:19.041 7f1b7bfcea20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 02:18:37.768 7f1b7c397f60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 02:18:54.366 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:19:02.797 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19859752, partition_space_available_1k=61230732, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1418036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11276, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51204, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8665472, free=488060, buffers=719612, cached=6302492, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.29, five_min_avg=2.96, fifteen_min_avg=3.62, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:19:54.494 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:20:01.454 7f1b7c3878d0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 02:20:03.819 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20952852, partition_space_available_1k=60137632, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1410380, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11240, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50496, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8885512, free=334072, buffers=703488, cached=6252564, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.62, five_min_avg=2.92, fifteen_min_avg=3.56, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:20:54.387 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:21:02.811 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20979056, partition_space_available_1k=60111428, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1428024, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13236, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50280, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9112432, free=363420, buffers=694800, cached=6004984, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.01, five_min_avg=3.61, fifteen_min_avg=3.76, cpu_share=13.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:21:54.411 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:22:04.838 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21001440, partition_space_available_1k=60089044, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1451768, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13288, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50124, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9254784, free=346088, buffers=689592, cached=5885172, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.36, five_min_avg=3.99, fifteen_min_avg=3.88, cpu_share=14.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:22:54.406 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:23:03.853 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21010344, partition_space_available_1k=60080140, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1474600, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13356, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50080, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9297464, free=354732, buffers=688104, cached=5835336, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.36, five_min_avg=4.97, fifteen_min_avg=4.22, cpu_share=13.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:23:54.442 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:24:03.837 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21040940, partition_space_available_1k=60049544, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1490380, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13416, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50044, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9298628, free=372320, buffers=686332, cached=5818356, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.28, five_min_avg=4.75, fifteen_min_avg=4.19, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:24:54.431 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:25:02.856 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21055728, partition_space_available_1k=60034756, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503528, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13516, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50040, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9324360, free=351352, buffers=685932, cached=5813992, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.21, five_min_avg=4.85, fifteen_min_avg=4.26, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:25:54.431 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:26:03.864 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21068168, partition_space_available_1k=60022316, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1517732, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13336, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49988, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9361136, free=354572, buffers=683136, cached=5776792, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.55, five_min_avg=4.65, fifteen_min_avg=4.23, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:26:54.395 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:26:59.497 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742437619497298, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=1, pending=1, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:27:04.904 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21079228, partition_space_available_1k=60011256, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1559136, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13360, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49920, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9318852, free=420208, buffers=681600, cached=5754976, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.85, five_min_avg=4.42, fifteen_min_avg=4.17, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:27:14.466 7f1b7c08b6d8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/20/2025 02:27:54.449 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:27:57.752 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2923-1.', _node_type='Database', _instance_id='0', _timestamp=1742437677752606, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/20/2025 02:28:02.872 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21118924, partition_space_available_1k=59971560, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1536424, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13516, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49936, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9207004, free=474480, buffers=682532, cached=5811620, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.8, five_min_avg=4.78, fifteen_min_avg=4.32, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:28:54.471 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/20/2025 02:29:02.874 7f1b7e78ef08 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2923-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21164376, partition_space_available_1k=59926108, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1536076, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13580, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49912, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9148436, free=306932, buffers=684008, cached=6036260, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.9, five_min_avg=4.22, fifteen_min_avg=4.16, cpu_share=9.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/20/2025 02:29:13.460 7f1b7beab2e8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map