06/23/2025 00:40:25.843 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 06/23/2025 00:40:25.843 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 06/23/2025 00:40:25.843 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 06/23/2025 00:40:25.851 7f88f9ae4148 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 06/23/2025 00:40:25.851 7f88f9ae4148 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 06/23/2025 00:40:25.852 7f88f6052648 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 06/23/2025 00:40:25.854 7f88f6052648 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 06/23/2025 00:40:25.855 7f88f6052648 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/23/2025 00:40:25.855 7f88f6052648 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 06/23/2025 00:40:25.855 7f88f6052648 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:25.855 7f88f6052648 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/23/2025 00:40:25.855 7f88f6052648 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:25.856 7f88f6052648 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:25.856 7f88f6052648 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 06/23/2025 00:40:25.856 7f88f6052648 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:25.910 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 27234100277305246539643629903907032529357216389697011142057973595688661650508, 'start': '1750639223000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 32563676875325625754550428768983200760391347894896176449536579526294759054993, 'start': '1750639223000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 31109567546425395437316241907067543439421382339852435192516771831498899270908, 'start': '1750639223000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 06/23/2025 00:40:26.068 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.068 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-06-16 01:01:35.123399", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9510-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 06/23/2025 00:40:26.068 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.069 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3515-3.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1750639226069405, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 06/23/2025 00:40:26.069 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3515-3.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1750639226069806, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1750639223000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1750639223000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1750639223000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-06-16 01:01:35.123399", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9510-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 06/23/2025 00:40:26.072 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.072 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/23/2025 00:40:26.072 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.072 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.072 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1750639223000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1750639223000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1750639223000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-06-16 01:01:35.123399", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9510-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 06/23/2025 00:40:26.072 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.295 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:40:26.505 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 06/23/2025 00:40:26.507 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.507 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. database_usage = [ << disk_space_used_1k = 4740236 disk_space_available_1k = 76350248 config_db_size_1k = 4 >>, ] >> 06/23/2025 00:40:26.507 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.889 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 06/23/2025 00:40:26.889 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3515-3.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1750639226889420, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 06/23/2025 00:40:26.891 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:26.891 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 06/23/2025 00:40:26.891 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:28.313 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:40:29.853 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:29.855 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:29.855 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:29.855 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:29.856 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:29.856 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:29.856 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:29.856 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:29.856 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:29.856 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:29.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:29.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:29.857 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:29.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:30.995 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 06/23/2025 00:40:32.497 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 06/23/2025 00:40:33.434 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 06/23/2025 00:40:33.856 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:33.857 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:33.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:33.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:33.857 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:33.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:33.857 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:33.858 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:34.414 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 06/23/2025 00:40:35.354 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 06/23/2025 00:40:36.278 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 06/23/2025 00:40:37.858 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:37.860 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:37.860 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:37.860 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:37.860 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:37.860 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:37.861 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:37.862 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.303 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3515-3.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1750639241303537, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1750639223000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1750639223000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1750639223000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-06-16 01:01:35.123399", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9510-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 06/23/2025 00:40:41.304 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=5091632, partition_space_available_1k=75998852, percentage_partition_space_used=6, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1282216, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=60640, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51072, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=1763472, free=9331700, buffers=100508, cached=4979956, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.69, five_min_avg=0.7, fifteen_min_avg=0.27, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:40:41.304 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.304 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1750639223000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1750639223000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1750639223000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-06-16 01:01:35.123399", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9510-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 06/23/2025 00:40:41.304 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.304 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.305 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 5091632 partition_space_available_1k = 75998852 percentage_partition_space_used = 6 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1282216 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 60640 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51072 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 1763472 free = 9331700 buffers = 100508 cached = 4979956 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.69 five_min_avg = 0.7 fifteen_min_avg = 0.27 cpu_share = 0 node_type = config-database-node >> >> 06/23/2025 00:40:41.305 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.860 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:41.862 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:41.862 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:41.862 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:41.862 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:41.863 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:41.863 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:41.863 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:41.863 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.863 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:41.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:41.864 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:41.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:45.862 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:45.864 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:45.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:45.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:45.864 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:45.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:45.864 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:45.865 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:45.865 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:45.865 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:45.865 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:45.865 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:45.865 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:45.866 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:49.867 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:49.869 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:49.870 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:49.870 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:49.870 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:49.870 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:49.870 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:49.870 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:53.869 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:53.871 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:53.871 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:53.871 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:53.872 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:53.872 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:53.872 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:53.873 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:53.873 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:53.874 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:53.874 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:53.874 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:53.874 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:53.874 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:57.872 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:40:57.874 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:40:57.874 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:40:57.874 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:57.875 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:40:57.875 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:40:57.875 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:40:57.875 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:40:57.876 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:57.876 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:40:57.876 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:57.876 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:40:57.876 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:40:57.876 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:01.877 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:01.879 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:01.880 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:01.880 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:01.881 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:01.881 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:01.881 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:01.882 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:01.882 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:01.882 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:01.882 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:01.883 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:01.883 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:01.883 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:05.881 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:05.883 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:05.883 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:05.883 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:05.883 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:05.883 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:05.884 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:05.885 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:09.885 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:09.887 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:09.887 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:09.887 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:09.887 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:09.887 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:09.887 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:09.888 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:09.888 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:09.888 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:09.888 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:09.889 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:09.889 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:09.889 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:13.889 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:13.891 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:13.891 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:13.891 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:13.892 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:13.893 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:13.893 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:17.894 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:17.896 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:17.896 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:17.897 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:17.897 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:17.897 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:17.897 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:17.898 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:17.899 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:17.899 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:17.899 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:17.900 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:17.900 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:17.901 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:21.899 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:21.901 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:21.901 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:21.901 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:21.901 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:21.901 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:21.901 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:21.902 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:25.901 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:25.903 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:25.903 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:25.904 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:25.904 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:25.904 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:25.905 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:25.905 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:25.905 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:25.905 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:25.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:25.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:25.906 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:25.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:26.437 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:41:26.775 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:26.775 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. database_usage = [ << disk_space_used_1k = 6062428 disk_space_available_1k = 75028056 config_db_size_1k = 1540 >>, ] >> 06/23/2025 00:41:26.776 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:27.418 7f88fb857df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3515-3.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1750639287418534, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.49:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 06/23/2025 00:41:27.420 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:27.420 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:27.425 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:28.649 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:41:29.904 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:29.905 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:29.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:29.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:29.906 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:29.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:29.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:29.907 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.906 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:33.908 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:33.909 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.909 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:33.909 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.909 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:33.909 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:33.909 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:34.258 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6127476, partition_space_available_1k=74963008, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341392, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71060, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51556, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=2296672, free=7877968, buffers=200472, cached=5800524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.27, five_min_avg=1.14, fifteen_min_avg=0.45, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:41:34.325 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:34.326 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6127476 partition_space_available_1k = 74963008 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1341392 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 71060 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51556 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 2296672 free = 7877968 buffers = 200472 cached = 5800524 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.27 five_min_avg = 1.14 fifteen_min_avg = 0.45 cpu_share = 11.25 node_type = config-database-node >> >> 06/23/2025 00:41:34.327 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.911 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:37.912 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:37.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.914 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:37.914 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.914 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:37.914 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:37.914 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.913 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:41.915 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:41.915 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.916 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:41.917 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.917 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:41.917 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:41.917 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.917 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:45.919 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:45.919 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:45.919 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:45.920 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:45.920 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:45.920 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:45.920 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:45.921 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.921 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:45.921 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.921 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:45.921 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:45.921 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.921 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:49.923 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:49.923 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:49.923 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:49.924 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.925 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:49.925 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:49.925 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.923 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:53.925 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:53.925 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:53.925 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:53.925 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:53.925 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:53.925 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:53.926 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.926 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:41:57.927 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:41:57.927 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:41:57.927 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.928 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:41:57.929 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:41:57.929 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.929 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:01.934 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:01.934 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:01.935 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:01.935 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:01.935 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:01.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:01.937 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:01.937 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.937 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:01.938 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.938 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:01.938 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:01.938 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.934 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:05.935 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:05.936 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.937 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:05.937 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.937 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:05.937 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:05.937 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.937 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:09.939 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:09.939 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:09.939 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:09.939 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:09.939 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:09.940 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:09.940 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:09.940 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.940 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:09.940 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.940 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:09.941 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:09.941 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.941 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:13.942 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:13.943 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:13.943 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:13.943 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:13.943 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:13.943 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:13.944 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.945 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:17.947 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:17.947 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:17.947 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:17.947 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:17.948 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.950 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:21.951 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:21.951 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:21.952 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:21.952 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:21.952 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:21.952 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:21.952 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:21.952 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.953 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:21.953 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.953 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:21.953 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:21.953 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.953 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:25.955 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:25.955 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:25.956 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:25.956 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:25.956 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:25.956 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:25.956 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:25.957 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.957 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:25.957 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.957 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:25.958 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:25.958 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:26.275 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:42:26.522 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:26.522 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. database_usage = [ << disk_space_used_1k = 6431040 disk_space_available_1k = 74659444 config_db_size_1k = 1736 >>, ] >> 06/23/2025 00:42:26.522 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:28.042 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:42:29.955 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:29.957 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:29.958 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:29.958 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:29.958 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:29.958 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:29.958 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:29.959 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.235 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6431064, partition_space_available_1k=74659420, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1649680, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84684, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51688, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=2982836, free=7066856, buffers=208476, cached=5917468, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=1.63, fifteen_min_avg=0.66, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:42:33.237 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.237 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6431064 partition_space_available_1k = 74659420 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1649680 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 84684 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51688 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 2982836 free = 7066856 buffers = 208476 cached = 5917468 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.58 five_min_avg = 1.63 fifteen_min_avg = 0.66 cpu_share = 8.5 node_type = config-database-node >> >> 06/23/2025 00:42:33.237 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.958 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:33.959 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:33.960 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:33.960 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:33.960 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:33.960 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:33.960 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:33.960 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:33.961 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.961 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:33.961 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.961 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:33.961 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:33.961 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.962 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:37.963 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:37.963 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:37.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.965 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:37.965 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:37.965 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.964 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:41.965 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:41.965 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:41.965 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:41.965 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:41.966 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.965 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:45.966 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.967 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:45.968 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.968 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:45.968 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:45.968 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.969 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:49.973 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:49.973 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:49.973 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:49.974 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:49.974 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:49.974 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:49.974 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:49.975 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.975 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:49.975 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.975 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:49.975 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:49.975 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.976 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:53.977 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:53.977 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:53.977 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:53.978 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:53.978 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:53.978 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:53.978 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:53.978 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.979 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:53.979 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.979 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:53.979 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:53.979 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.981 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:42:57.982 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:42:57.983 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:42:57.983 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:57.983 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:42:57.983 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:42:57.983 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:42:57.984 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:42:57.984 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.984 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:42:57.984 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.984 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:42:57.984 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:42:57.985 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.983 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:01.985 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:01.985 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:01.985 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:01.985 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:01.985 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:01.985 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:01.986 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:01.986 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.986 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:01.986 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.987 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:01.987 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:01.987 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.986 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:05.987 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:05.987 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:05.988 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:05.988 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:05.988 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:05.988 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:05.988 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:05.988 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.989 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:05.989 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.989 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:05.989 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:05.989 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.991 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:09.992 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:09.993 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.994 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:09.994 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.994 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:09.994 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:09.994 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.994 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:13.995 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:13.996 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.997 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:13.997 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:13.997 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:17.998 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:18.000 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:18.000 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:18.000 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:18.000 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:18.000 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:18.000 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:18.001 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:18.001 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.001 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:18.001 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.001 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:18.002 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:18.002 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:22.001 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:22.003 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:22.004 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:22.004 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:22.004 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:22.004 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:22.004 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:22.004 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:22.005 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:22.005 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:22.005 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:22.005 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:22.005 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:22.005 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:26.006 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:26.007 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:26.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:26.009 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:26.009 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:26.009 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:26.009 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:26.009 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:26.291 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:43:26.549 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:26.549 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. database_usage = [ << disk_space_used_1k = 6834056 disk_space_available_1k = 74256428 config_db_size_1k = 1808 >>, ] >> 06/23/2025 00:43:26.549 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:28.020 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:43:30.008 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:30.011 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:30.011 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:30.011 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:30.011 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:30.011 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:30.011 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:30.012 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:30.012 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:30.012 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:30.012 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:30.012 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:30.013 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:30.013 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:33.129 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6833972, partition_space_available_1k=74256512, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1732964, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=91220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52260, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4404628, free=5160548, buffers=238216, cached=6372244, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.87, five_min_avg=2.03, fifteen_min_avg=0.86, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:43:33.130 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:33.130 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6833972 partition_space_available_1k = 74256512 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1732964 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 91220 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 52260 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4404628 free = 5160548 buffers = 238216 cached = 6372244 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.87 five_min_avg = 2.03 fifteen_min_avg = 0.86 cpu_share = 8.75 node_type = config-database-node >> >> 06/23/2025 00:43:33.130 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:34.011 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:34.012 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:34.012 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:34.013 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:34.013 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:34.013 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:34.013 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:34.014 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:38.015 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:38.016 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:38.017 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:38.017 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:38.017 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:38.017 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:38.017 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:38.018 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:42.020 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:42.022 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:42.022 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:42.022 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:42.022 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:42.022 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:42.022 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:42.023 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:42.023 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:42.023 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:42.023 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:42.023 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:42.023 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:42.024 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:46.025 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:46.026 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:46.026 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:46.026 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:46.026 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:46.026 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:46.026 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:46.027 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:50.029 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:50.031 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 06/23/2025 00:43:50.032 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:50.032 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:50.032 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:50.032 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:50.032 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 06/23/2025 00:43:50.032 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 06/23/2025 00:43:50.033 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:50.033 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 06/23/2025 00:43:50.033 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:50.033 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:50.033 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1750639225855086 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.19:8086, 10.0.0.41:8086, 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 06/23/2025 00:43:50.033 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 06/23/2025 00:43:54.033 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3515-3. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 06/23/2025 00:43:54.035 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 06/23/2025 00:43:54.036 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 06/23/2025 00:43:54.036 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/23/2025 00:43:54.036 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 06/23/2025 00:44:05.662 7f88f5ed3048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 06/23/2025 00:44:05.663 7f88f5ed3048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/23/2025 00:44:05.663 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 06/23/2025 00:44:05.664 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 06/23/2025 00:44:05.664 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 06/23/2025 00:44:05.664 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 06/23/2025 00:44:05.664 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/23/2025 00:44:05.664 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 06/23/2025 00:44:05.664 7f88f6052e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/23/2025 00:44:05.666 7f88f6052e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/23/2025 00:44:16.090 7f88f5ed3048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 06/23/2025 00:44:16.090 7f88f5ed3048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 06/23/2025 00:44:16.091 7f88f6052e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 06/23/2025 00:44:16.091 7f88f6052e48 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 06/23/2025 00:44:16.091 7f88f6052e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 06/23/2025 00:44:16.091 7f88f6052e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 06/23/2025 00:44:16.091 7f88f6052e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 06/23/2025 00:44:16.092 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 06/23/2025 00:44:16.092 7f88f5ed3348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 06/23/2025 00:44:26.398 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:44:28.700 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:44:33.978 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7250004, partition_space_available_1k=73840480, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1731672, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=164624, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52436, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5616776, free=3457808, buffers=261112, cached=6839940, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.21, five_min_avg=2.35, fifteen_min_avg=1.04, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:45:26.355 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:45:28.460 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:45:33.992 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7316528, partition_space_available_1k=73773956, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1730604, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=165276, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52448, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5653772, free=3349428, buffers=262788, cached=6909648, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=2.63, fifteen_min_avg=1.21, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:46:26.407 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:46:28.410 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:46:33.994 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7356860, partition_space_available_1k=73733624, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1730340, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150592, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52456, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5666032, free=3285956, buffers=263764, cached=6959884, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.9, five_min_avg=2.54, fifteen_min_avg=1.27, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:46:43.042 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 06/23/2025 00:47:02.237 7f88f5ed3348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 06/23/2025 00:47:26.366 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:47:28.302 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:47:34.009 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7814204, partition_space_available_1k=73276280, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1733228, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147000, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52508, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5686592, free=2790044, buffers=270076, cached=7428924, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.66, five_min_avg=2.79, fifteen_min_avg=1.44, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:48:26.396 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:48:28.352 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:48:34.022 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7823196, partition_space_available_1k=73267288, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1734024, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152316, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52516, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5696884, free=2775016, buffers=271172, cached=7432564, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.13, five_min_avg=2.79, fifteen_min_avg=1.52, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:49:26.434 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:49:28.528 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:49:34.039 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7818240, partition_space_available_1k=73272244, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1800068, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=157396, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52580, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5778440, free=2691352, buffers=272968, cached=7432876, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.27, five_min_avg=2.85, fifteen_min_avg=1.62, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:50:26.389 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:50:28.616 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:50:34.040 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7832856, partition_space_available_1k=73257628, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1854196, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=163084, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52600, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5831660, free=2626820, buffers=274228, cached=7442928, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.71, five_min_avg=3.0, fifteen_min_avg=1.75, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:51:26.430 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:51:28.515 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:51:34.050 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7850140, partition_space_available_1k=73240344, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1876184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170284, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52608, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5897920, free=2551420, buffers=275020, cached=7451276, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.27, five_min_avg=2.99, fifteen_min_avg=1.82, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:52:26.470 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:52:28.729 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:52:34.085 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7855976, partition_space_available_1k=73234508, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1895500, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=175592, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52612, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5919632, free=2517904, buffers=275600, cached=7462500, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=2.97, fifteen_min_avg=1.89, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:53:26.406 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:53:28.427 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:53:34.080 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7866488, partition_space_available_1k=73223996, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1912320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=180608, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52628, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5941824, free=2488308, buffers=276216, cached=7469288, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.82, five_min_avg=2.93, fifteen_min_avg=1.95, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:54:26.415 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:54:28.425 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:54:34.089 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7873340, partition_space_available_1k=73217144, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1952416, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185708, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52636, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5989012, free=2428460, buffers=276780, cached=7481384, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.62, five_min_avg=2.86, fifteen_min_avg=1.99, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:55:26.442 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:55:28.442 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:55:34.077 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7884804, partition_space_available_1k=73205680, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1977624, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=190564, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52644, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6030592, free=2379380, buffers=277120, cached=7488544, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.14, five_min_avg=2.97, fifteen_min_avg=2.08, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:56:26.456 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:56:28.499 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:56:34.091 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7892116, partition_space_available_1k=73198368, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2007816, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=195724, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52644, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6026136, free=2370572, buffers=277432, cached=7501496, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=3.01, fifteen_min_avg=2.15, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:57:26.443 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:57:28.424 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:57:34.157 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7900860, partition_space_available_1k=73189624, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2037132, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=201216, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52672, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6082088, free=2304912, buffers=277584, cached=7511052, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.06, fifteen_min_avg=2.22, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:58:26.434 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:58:28.389 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:58:34.101 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7909668, partition_space_available_1k=73180816, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2086808, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=206600, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52660, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6142872, free=2229708, buffers=277768, cached=7525288, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=3.02, fifteen_min_avg=2.26, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 00:59:26.421 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 00:59:28.316 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 00:59:34.122 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7918116, partition_space_available_1k=73172368, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2143932, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=211672, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52664, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6203484, free=2159400, buffers=277836, cached=7534916, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.16, five_min_avg=3.06, fifteen_min_avg=2.32, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:00:26.445 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:00:28.341 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:00:34.122 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7925844, partition_space_available_1k=73164640, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2163440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=216816, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52668, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6238408, free=2111816, buffers=277892, cached=7547520, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=3.25, fifteen_min_avg=2.43, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:01:26.452 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:01:28.486 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:01:34.123 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7942676, partition_space_available_1k=73147808, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2175996, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=221908, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52672, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6237352, free=2104852, buffers=277944, cached=7555488, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.36, five_min_avg=3.23, fifteen_min_avg=2.48, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:02:26.436 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:02:28.372 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:02:34.130 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7951240, partition_space_available_1k=73139244, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2190328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=227384, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52676, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6266632, free=2061152, buffers=277976, cached=7569876, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.61, five_min_avg=3.56, fifteen_min_avg=2.64, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:03:26.440 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:03:28.425 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:03:34.135 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7958904, partition_space_available_1k=73131580, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2195924, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=232672, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52680, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6275508, free=2044316, buffers=278008, cached=7577804, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.62, five_min_avg=3.44, fifteen_min_avg=2.65, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:04:26.460 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:04:28.431 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:04:34.146 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7965588, partition_space_available_1k=73124896, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2198124, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=240868, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52680, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6284812, free=2046776, buffers=278016, cached=7566032, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=3.25, fifteen_min_avg=2.63, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:05:26.455 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:05:28.374 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:05:34.156 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7954008, partition_space_available_1k=73136476, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2179244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=246232, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52684, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6291836, free=2032496, buffers=278024, cached=7573280, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.14, fifteen_min_avg=2.64, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:06:26.464 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:06:28.344 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:06:34.151 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7961056, partition_space_available_1k=73129428, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2182836, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=251532, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52692, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6305792, free=2011352, buffers=278032, cached=7580460, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.88, five_min_avg=3.08, fifteen_min_avg=2.64, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:07:26.460 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:07:28.450 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:07:34.142 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7974112, partition_space_available_1k=73116372, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2191824, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=256828, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52692, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6318924, free=1990120, buffers=278036, cached=7588556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.08, five_min_avg=3.11, fifteen_min_avg=2.68, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:08:26.460 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:08:28.274 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:08:34.154 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7976644, partition_space_available_1k=73113840, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2199624, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262188, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52700, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6338304, free=1957732, buffers=278052, cached=7601548, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.41, five_min_avg=3.17, fifteen_min_avg=2.73, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:09:26.456 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:09:28.392 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:09:34.170 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7983852, partition_space_available_1k=73106632, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2199988, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=267732, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52700, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6341100, free=1947824, buffers=278068, cached=7608644, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.92, five_min_avg=3.27, fifteen_min_avg=2.79, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:10:26.450 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:10:28.429 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:10:34.167 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7996088, partition_space_available_1k=73094396, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2204356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=272896, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52704, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6353352, free=1932632, buffers=278068, cached=7611584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.14, five_min_avg=3.17, fifteen_min_avg=2.79, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:11:26.472 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:11:28.547 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:11:34.149 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8003112, partition_space_available_1k=73087372, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2204308, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=278732, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53104, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6349368, free=1925000, buffers=278080, cached=7623188, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=3.06, fifteen_min_avg=2.77, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:12:26.546 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:12:28.646 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:12:34.159 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8010136, partition_space_available_1k=73080348, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2206000, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=284036, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53112, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6364664, free=1902692, buffers=278084, cached=7630196, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.73, five_min_avg=3.24, fifteen_min_avg=2.85, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:13:26.505 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:13:28.465 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:13:34.160 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8015592, partition_space_available_1k=73074892, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2205836, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=289444, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53116, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6362780, free=1898376, buffers=278084, cached=7636396, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.82, five_min_avg=3.06, fifteen_min_avg=2.81, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:14:26.500 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:14:28.556 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:14:34.168 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8021468, partition_space_available_1k=73069016, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2205656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=289972, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53116, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6372172, free=1883088, buffers=278084, cached=7642292, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=3.24, fifteen_min_avg=2.89, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:15:26.509 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:15:28.484 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:15:34.153 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8027260, partition_space_available_1k=73063224, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2206116, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=295308, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53120, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6383948, free=1865644, buffers=278084, cached=7647960, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=3.14, fifteen_min_avg=2.88, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:16:26.505 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:16:28.427 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:16:34.169 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8037288, partition_space_available_1k=73053196, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2208280, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=300784, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53128, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6352568, free=1891220, buffers=278084, cached=7653764, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.69, five_min_avg=3.0, fifteen_min_avg=2.85, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:17:26.537 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:17:28.408 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:17:34.157 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8044248, partition_space_available_1k=73046236, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2218848, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=305908, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53148, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6377820, free=1858736, buffers=278088, cached=7660992, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.0, five_min_avg=3.01, fifteen_min_avg=2.86, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:18:26.551 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:18:28.563 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:18:34.174 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8050704, partition_space_available_1k=73039780, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2209324, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=311908, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53152, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6383120, free=1846824, buffers=278088, cached=7667604, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=3.0, fifteen_min_avg=2.87, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:19:26.619 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:19:28.584 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:19:34.187 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8051916, partition_space_available_1k=73038568, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2222320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=317340, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53152, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6394168, free=1829696, buffers=278088, cached=7673684, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.19, five_min_avg=3.05, fifteen_min_avg=2.89, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:20:26.573 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:20:28.522 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:20:34.182 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8057704, partition_space_available_1k=73032780, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2209424, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=322764, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53164, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6396296, free=1816468, buffers=278092, cached=7684780, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=2.97, fifteen_min_avg=2.88, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:21:26.579 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:21:28.587 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:21:34.214 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8069000, partition_space_available_1k=73021484, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2209852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=328400, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53168, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6405432, free=1800680, buffers=278092, cached=7691432, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.34, five_min_avg=3.09, fifteen_min_avg=2.92, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:22:26.586 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:22:28.558 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:22:34.214 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8088324, partition_space_available_1k=73002160, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2211560, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=334200, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53180, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6374272, free=1812152, buffers=278096, cached=7711116, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=2.96, fifteen_min_avg=2.89, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:23:26.599 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:23:28.541 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:23:34.216 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8095496, partition_space_available_1k=72994988, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2211808, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=339680, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53176, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6384620, free=1794548, buffers=278096, cached=7718372, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.6, five_min_avg=2.89, fifteen_min_avg=2.87, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:24:26.609 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:24:28.596 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:24:34.240 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8101940, partition_space_available_1k=72988544, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2211148, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=345236, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53180, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6359832, free=1812768, buffers=278100, cached=7724936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.76, five_min_avg=3.19, fifteen_min_avg=2.98, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:25:26.613 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:25:28.576 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:25:34.248 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8107536, partition_space_available_1k=72982948, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2212456, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=350660, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53184, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6378660, free=1783060, buffers=278100, cached=7735816, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.84, five_min_avg=3.03, fifteen_min_avg=2.94, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:26:26.600 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:26:28.639 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:26:34.255 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8113164, partition_space_available_1k=72977320, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2212428, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=356124, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53188, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6389180, free=1771644, buffers=278100, cached=7736712, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.44, five_min_avg=3.13, fifteen_min_avg=2.97, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:27:26.611 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:27:28.582 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:27:34.263 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8124140, partition_space_available_1k=72966344, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214976, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=361720, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53196, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6403868, free=1755400, buffers=278100, cached=7738268, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=2.91, fifteen_min_avg=2.91, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:28:26.659 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:28:28.623 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:28:34.263 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8130428, partition_space_available_1k=72960056, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214636, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=367628, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53196, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6405964, free=1742172, buffers=278100, cached=7749400, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.69, five_min_avg=2.88, fifteen_min_avg=2.9, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:29:26.655 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:29:28.666 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:29:34.281 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8131980, partition_space_available_1k=72958504, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=373328, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53196, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6410356, free=1731052, buffers=278104, cached=7756124, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.57, five_min_avg=2.79, fifteen_min_avg=2.86, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:30:26.627 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:30:28.633 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:30:34.279 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8142612, partition_space_available_1k=72947872, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214856, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=378880, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53200, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6414516, free=1721384, buffers=278104, cached=7761632, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.63, five_min_avg=2.77, fifteen_min_avg=2.85, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:31:26.627 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:31:28.584 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:31:34.286 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8148424, partition_space_available_1k=72942060, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2215096, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=384456, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53204, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6417404, free=1707400, buffers=278104, cached=7772728, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.51, five_min_avg=2.69, fifteen_min_avg=2.82, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:32:26.628 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:32:28.594 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:32:34.293 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8149060, partition_space_available_1k=72941424, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216956, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=390028, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53212, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6430376, free=1693416, buffers=278108, cached=7773736, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.67, five_min_avg=2.69, fifteen_min_avg=2.81, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:33:26.626 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:33:28.601 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:33:34.302 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8162880, partition_space_available_1k=72927604, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2217924, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=395920, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53216, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6431704, free=1688200, buffers=278108, cached=7777624, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=2.75, fifteen_min_avg=2.83, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:34:26.655 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 06/23/2025 01:34:28.673 7f88fb857df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 06/23/2025 01:34:34.302 7f88fb857df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3515-3.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8166608, partition_space_available_1k=72923876, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216644, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=401760, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53224, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6433596, free=1672560, buffers=278108, cached=7791372, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.46, five_min_avg=2.64, fifteen_min_avg=2.78, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 06/23/2025 01:34:42.516 7f88f6052e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map