02/06/2025 00:46:30.280 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 00:46:30.280 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 00:46:30.280 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 00:46:30.288 7f37afea9648 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 02/06/2025 00:46:30.288 7f37afea9648 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 02/06/2025 00:46:30.289 7f37afea9e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 00:46:30.290 7f37afea9e48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:46:30.290 7f37afea9e48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 00:46:30.291 7f37afea9e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.478 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 8791581627069423503242437477526788983655632461857104008600290540168351363478, 'start': '1738802787000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 30537933908399769734425062145851391525749299239785097784796206822473691264719, 'start': '1738802787000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 19493228031657763537858731289839985755418907111729163643279168634164941493756, 'start': '1738802787000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 00:46:30.669 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.670 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/06/2025 00:46:30.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.670 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738802790670670, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 00:46:30.671 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738802790670992, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 00:46:30.674 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.674 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:46:30.674 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.674 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:30.674 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738802787000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738802787000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738802787000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 00:46:30.674 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:31.005 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:46:31.678 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 00:46:31.680 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:31.680 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 11698432 disk_space_available_1k = 69392052 config_db_size_1k = 4 >>, ] >> 02/06/2025 00:46:31.681 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:32.402 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 00:46:32.403 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738802792403029, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 00:46:32.416 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:32.416 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:46:32.416 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:34.290 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:34.291 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:34.291 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:34.292 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:34.293 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:34.293 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:34.293 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:34.293 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:35.378 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:46:37.194 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 02/06/2025 00:46:38.226 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 02/06/2025 00:46:38.292 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:38.293 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:38.293 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:38.294 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:38.294 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:38.294 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:38.294 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:38.294 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:38.295 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:38.295 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:38.295 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:38.295 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:38.295 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:38.295 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:40.508 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 02/06/2025 00:46:41.823 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 02/06/2025 00:46:42.294 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:42.296 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:42.296 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:42.296 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:42.296 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:42.296 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:42.296 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:42.297 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:42.996 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 02/06/2025 00:46:43.863 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 02/06/2025 00:46:46.296 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:46.298 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:46.298 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:46.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:46.300 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:46.300 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:46.300 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:48.891 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738802808891093, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 00:46:48.891 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12050760, partition_space_available_1k=69039724, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1236900, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57564, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51096, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4550968, free=293300, buffers=536520, cached=10794848, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.58, five_min_avg=2.29, fifteen_min_avg=1.35, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:46:48.891 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:48.891 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738802787000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738802787000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738802787000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 00:46:48.892 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:48.892 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:48.892 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12050760 partition_space_available_1k = 69039724 percentage_partition_space_used = 15 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1236900 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 57564 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51096 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4550968 free = 293300 buffers = 536520 cached = 10794848 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 4.58 five_min_avg = 2.29 fifteen_min_avg = 1.35 cpu_share = 0 node_type = config-database-node >> >> 02/06/2025 00:46:48.892 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:50.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:50.300 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:50.301 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:50.301 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:50.301 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:50.301 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:50.301 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:50.302 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:54.304 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:54.305 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:54.305 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:54.305 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:54.306 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:54.307 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:54.307 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:54.307 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:58.308 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:46:58.309 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:46:58.309 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:46:58.309 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:58.309 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:46:58.309 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:46:58.309 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:46:58.310 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:46:58.310 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:58.310 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:46:58.310 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:58.310 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:46:58.311 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:46:58.311 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:02.309 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:02.311 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:02.311 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:02.311 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:02.311 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:02.311 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:02.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:06.312 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:06.313 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:06.313 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:06.313 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:06.313 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:06.314 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:06.315 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:06.315 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:10.316 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:10.445 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:10.446 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:10.446 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:10.446 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:10.446 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:10.446 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:10.446 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:10.447 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:10.447 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:10.447 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:10.447 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:10.447 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:10.447 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:14.448 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:14.449 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:14.449 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:14.449 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:14.451 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:14.451 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:14.451 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:14.451 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:14.452 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:14.452 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:14.452 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:14.452 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:14.452 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:14.452 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:18.451 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:18.453 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:18.454 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:18.454 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:18.454 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:18.454 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:18.454 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:18.454 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:18.455 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:18.455 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:18.455 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:18.455 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:18.455 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:18.455 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:22.456 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:22.464 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:22.464 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:22.464 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:22.465 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:22.465 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:22.465 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:22.465 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:22.465 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:22.466 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:22.466 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:22.466 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:22.466 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:22.467 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:26.468 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:26.470 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:26.470 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:26.470 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:26.470 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:26.471 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:26.471 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:26.472 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:26.472 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:26.472 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:26.473 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:26.473 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:26.473 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:26.473 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:30.472 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:30.473 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:30.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:30.475 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:30.475 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:30.475 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:30.475 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:30.475 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:30.979 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:47:31.420 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:31.436 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 12974152 disk_space_available_1k = 68116332 config_db_size_1k = 1512 >>, ] >> 02/06/2025 00:47:31.436 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:32.468 7f37b583bdf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738802852468123, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 00:47:32.470 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:32.470 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:32.470 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:33.614 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:47:34.474 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:34.476 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:34.476 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:34.476 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:34.477 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:34.477 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:34.477 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:34.478 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:34.478 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:34.479 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:34.479 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:34.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:34.483 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:34.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:38.477 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:38.479 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:38.479 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:38.479 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:38.479 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:38.479 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:38.479 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:38.480 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:38.639 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12974192, partition_space_available_1k=68116292, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1912388, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57400, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51060, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5196288, free=296800, buffers=597572, cached=10084976, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.54, five_min_avg=2.95, fifteen_min_avg=1.63, cpu_share=18.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:47:38.639 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:38.639 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12974192 partition_space_available_1k = 68116292 percentage_partition_space_used = 16 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1912388 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 57400 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51060 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 5196288 free = 296800 buffers = 597572 cached = 10084976 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.54 five_min_avg = 2.95 fifteen_min_avg = 1.63 cpu_share = 18.75 node_type = config-database-node >> >> 02/06/2025 00:47:38.639 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:42.480 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:42.482 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:42.482 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:42.482 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:42.482 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:42.482 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:42.482 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:42.483 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:46.485 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:46.486 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:46.486 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:46.486 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:46.487 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:46.488 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:46.488 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:50.488 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:50.490 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:50.490 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:50.490 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:50.490 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:50.490 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:50.491 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:50.492 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:54.493 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:54.495 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:54.495 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:54.495 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:54.495 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:54.495 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:54.495 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:54.496 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:58.498 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:47:58.499 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:47:58.500 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:47:58.500 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:58.500 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:47:58.500 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:47:58.500 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:47:58.501 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:02.500 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:02.502 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:02.503 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:02.503 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:02.503 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:02.503 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:02.503 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:02.503 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:06.505 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:06.506 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:06.506 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:06.506 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:06.506 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:06.507 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:06.507 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:06.507 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:06.508 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:06.508 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:06.508 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:06.508 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:06.508 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:06.508 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:10.508 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:10.512 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:10.520 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:10.520 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:10.520 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:10.520 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:10.520 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:10.521 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:10.523 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:10.523 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:10.524 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:10.524 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:10.524 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:10.524 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:14.514 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:14.515 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:14.516 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:14.516 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:14.516 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:14.516 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:14.516 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:14.516 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:14.517 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:14.517 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:14.517 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:14.517 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:14.517 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:14.517 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:18.516 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:18.517 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:18.517 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:18.518 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:18.519 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:18.519 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:18.519 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:18.519 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:22.528 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:22.529 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:22.529 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:22.529 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:22.530 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:22.530 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:22.530 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:22.530 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:22.530 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:22.540 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:22.540 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:22.541 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:22.541 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:22.541 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:26.532 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:26.538 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:26.538 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:26.539 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:26.540 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:26.540 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:26.540 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:26.540 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:30.541 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:30.542 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:30.542 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:30.543 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:30.543 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:30.543 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:30.544 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:30.544 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:30.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:30.545 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:30.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:30.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:30.546 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:30.546 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:31.237 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:48:31.938 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:31.942 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 13272496 disk_space_available_1k = 67817988 config_db_size_1k = 1616 >>, ] >> 02/06/2025 00:48:31.948 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:34.543 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:34.544 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:34.545 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:34.546 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:34.546 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:34.546 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:34.546 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:34.546 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:36.103 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:48:38.547 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:38.549 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:38.549 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:38.549 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:38.549 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:38.549 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:38.549 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:38.550 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:38.550 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:38.550 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:38.550 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:38.550 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:38.550 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:38.551 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:41.135 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13273080, partition_space_available_1k=67817404, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1948820, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75448, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5708940, free=328052, buffers=622308, cached=9516336, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.07, five_min_avg=3.51, fifteen_min_avg=1.9, cpu_share=19.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:48:41.136 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:41.136 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13273080 partition_space_available_1k = 67817404 percentage_partition_space_used = 16 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1948820 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 75448 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 50884 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 5708940 free = 328052 buffers = 622308 cached = 9516336 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 6.07 five_min_avg = 3.51 fifteen_min_avg = 1.9 cpu_share = 19.75 node_type = config-database-node >> >> 02/06/2025 00:48:41.136 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:42.550 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:42.551 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:42.551 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:42.551 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:42.552 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:42.552 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:42.552 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:42.552 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:42.552 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:42.553 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:42.553 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:42.553 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:42.553 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:42.553 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:46.555 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:46.556 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:46.556 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:46.556 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:46.557 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:46.558 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:46.558 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:46.558 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:50.557 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:50.564 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:50.564 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:50.564 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:50.564 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:50.564 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:50.565 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:50.565 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:50.565 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:50.566 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:50.566 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:50.566 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:50.566 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:50.567 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:54.564 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:54.565 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:54.565 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:54.566 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:54.566 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:54.566 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:54.566 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:54.567 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:54.567 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:54.567 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:54.567 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:54.567 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:54.568 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:54.568 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:58.568 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:48:58.569 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:48:58.569 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:48:58.569 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:58.569 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:48:58.569 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:48:58.569 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:48:58.570 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:02.581 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:02.596 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:02.596 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:02.596 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:02.596 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:02.596 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:02.596 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:02.597 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:06.597 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:06.598 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:06.598 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:06.599 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:06.599 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:06.599 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:06.600 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:06.600 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:06.600 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:06.601 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:06.601 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:06.601 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:06.601 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:06.602 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:10.600 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:10.601 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:10.601 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:10.601 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:10.602 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:10.602 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:10.602 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:10.603 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:10.603 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:10.603 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:10.603 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:10.603 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:10.604 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:10.604 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:14.604 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:14.605 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:14.605 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:14.605 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:14.606 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:14.607 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:14.607 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:14.607 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:18.608 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:18.616 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:18.616 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:18.616 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:18.616 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:18.616 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:18.616 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:18.617 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:18.617 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:18.617 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:18.617 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:18.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:18.618 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:18.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:22.616 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:22.617 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:22.617 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:22.617 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:22.617 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:22.618 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:22.619 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:22.619 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:26.620 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:26.621 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:26.621 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:26.621 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:26.621 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:26.622 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:26.623 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:26.623 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:30.625 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:30.626 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:30.626 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:30.626 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:30.626 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:30.626 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:30.627 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:30.628 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:31.035 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:49:31.573 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:31.573 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 13391220 disk_space_available_1k = 67699264 config_db_size_1k = 1624 >>, ] >> 02/06/2025 00:49:31.573 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:34.120 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:49:34.627 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:34.628 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:34.629 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:34.629 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:34.629 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:34.629 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:34.629 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:34.629 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:34.630 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:34.630 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:34.630 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:34.630 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:34.630 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:34.630 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:38.630 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:38.631 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:38.631 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:38.631 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:38.632 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:38.633 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:38.633 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:38.633 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:39.145 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13391128, partition_space_available_1k=67699356, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2050180, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78356, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50832, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6006528, free=358004, buffers=623656, cached=9187448, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.78, five_min_avg=3.9, fifteen_min_avg=2.14, cpu_share=21.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:49:39.145 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:39.145 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13391128 partition_space_available_1k = 67699356 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2050180 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 78356 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 50832 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 6006528 free = 358004 buffers = 623656 cached = 9187448 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.78 five_min_avg = 3.9 fifteen_min_avg = 2.14 cpu_share = 21.0 node_type = config-database-node >> >> 02/06/2025 00:49:39.146 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:42.633 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:42.635 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:42.635 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:42.635 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:42.635 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:42.635 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:42.635 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:42.636 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:46.638 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:46.639 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:46.639 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:46.640 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:46.641 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:46.641 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:46.641 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:46.641 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:50.640 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:50.641 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:50.641 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:50.641 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:50.642 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:50.643 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:50.643 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:50.643 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:54.649 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:54.651 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:54.651 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:54.651 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:54.651 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:54.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:54.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:54.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:54.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:54.653 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:54.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:54.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:54.653 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:54.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:58.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:49:58.653 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:49:58.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:49:58.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:58.653 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:49:58.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:49:58.653 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:49:58.654 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:02.656 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:02.657 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:02.657 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:02.657 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:02.658 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:02.659 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:02.659 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:06.660 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:06.661 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:06.661 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:06.661 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:06.662 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:06.663 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:06.663 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:10.667 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:10.669 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:10.669 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:10.669 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:10.669 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:10.669 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:10.670 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:14.673 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:14.675 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:14.675 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:14.675 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:14.675 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:14.675 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:14.676 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:14.676 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:14.677 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:14.677 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:14.677 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:14.678 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:14.678 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:14.678 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:18.692 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:18.706 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:18.706 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:18.706 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:18.706 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:18.707 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:18.707 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:18.707 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:18.707 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:18.708 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:18.708 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:18.708 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:18.708 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:18.708 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:22.709 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:22.710 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:22.710 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:22.710 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:22.710 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:22.710 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:22.710 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:22.711 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:26.712 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:26.713 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:26.713 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:26.713 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:26.713 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:26.713 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:26.714 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:30.717 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:30.719 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:30.719 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:30.719 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:30.720 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:30.720 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:30.720 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:30.720 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:30.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:30.721 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:30.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:30.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:30.721 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:30.724 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:31.059 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:50:31.558 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:31.558 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 13780660 disk_space_available_1k = 67309824 config_db_size_1k = 1624 >>, ] >> 02/06/2025 00:50:31.558 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:33.925 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:50:34.719 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:34.721 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:34.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:34.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:34.721 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:34.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:34.721 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:34.722 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:34.722 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:34.722 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:34.722 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:34.723 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:34.723 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:34.723 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:38.722 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:38.724 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:38.724 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:38.724 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:38.724 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:38.724 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:38.724 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:38.725 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:38.953 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13780816, partition_space_available_1k=67309668, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2047952, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78584, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50908, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6169992, free=389140, buffers=649004, cached=8967500, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.87, five_min_avg=4.01, fifteen_min_avg=2.29, cpu_share=16.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:50:38.953 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:38.953 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13780816 partition_space_available_1k = 67309668 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2047952 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 78584 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 50908 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 6169992 free = 389140 buffers = 649004 cached = 8967500 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 4.87 five_min_avg = 4.01 fifteen_min_avg = 2.29 cpu_share = 16.75 node_type = config-database-node >> >> 02/06/2025 00:50:38.954 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:42.725 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:42.726 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:42.726 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:42.726 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:42.727 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:42.728 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:42.728 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:42.728 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:46.728 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:46.729 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:46.729 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:46.729 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:46.729 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:46.729 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:46.729 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:46.730 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:50.730 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:50.731 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:50.731 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:50.731 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:50.732 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:50.733 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:50.733 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:50.733 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.732 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:54.733 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:54.733 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:54.733 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:54.733 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:54.733 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.734 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:54.735 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.736 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:58.737 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:58.737 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:58.737 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:58.737 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:58.737 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:58.737 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:58.738 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.740 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:02.741 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:02.742 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:02.742 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:02.742 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:02.742 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:02.742 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:02.744 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:02.744 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.746 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:02.746 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.746 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.747 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:02.747 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.742 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:06.743 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:06.743 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:06.743 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:06.744 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:06.744 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:06.744 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:06.744 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:06.744 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.744 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:06.745 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.745 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.745 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:06.745 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.747 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:10.748 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:10.748 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:10.748 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:10.748 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:10.748 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:10.748 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:10.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.748 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:14.749 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:14.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:14.749 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:14.750 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:14.750 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:14.750 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:14.750 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:14.750 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.751 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:14.751 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.751 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.751 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:14.752 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.752 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:18.753 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:18.753 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:18.754 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:18.754 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:18.754 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:18.754 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:18.755 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:18.755 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.755 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:18.755 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.756 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.759 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:18.759 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.756 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:22.757 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:22.757 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:22.757 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:22.757 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.758 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.759 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:22.759 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.760 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:26.761 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:26.761 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:26.761 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.762 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:26.763 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.769 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:30.774 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:30.775 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:30.775 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:30.775 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:30.775 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:30.775 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:30.775 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:30.776 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.776 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:30.776 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.776 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.776 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:30.776 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.987 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:51:31.382 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:31.382 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 13793996 disk_space_available_1k = 67296488 config_db_size_1k = 1624 >>, ] >> 02/06/2025 00:51:31.382 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:33.267 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:51:34.775 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:34.776 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:34.777 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:34.777 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:34.777 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:34.777 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:34.777 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:34.778 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:34.778 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.778 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:34.779 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.779 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.779 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:34.779 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.298 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13794024, partition_space_available_1k=67296460, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2046264, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80044, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50776, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7524560, free=351416, buffers=651156, cached=7648504, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.47, five_min_avg=4.33, fifteen_min_avg=2.5, cpu_share=18.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:51:38.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.299 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13794024 partition_space_available_1k = 67296460 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2046264 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 80044 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 50776 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7524560 free = 351416 buffers = 651156 cached = 7648504 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.47 five_min_avg = 4.33 fifteen_min_avg = 2.5 cpu_share = 18.75 node_type = config-database-node >> >> 02/06/2025 00:51:38.299 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.777 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:38.778 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:38.779 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:38.779 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:38.780 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:38.780 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:38.780 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:38.781 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:38.781 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.781 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:38.781 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.782 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.782 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:38.782 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.780 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:42.782 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:42.782 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:42.782 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:42.782 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:42.782 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:42.782 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:42.783 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:46.785 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:46.786 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 00:51:46.787 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:46.787 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:46.787 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:46.788 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:46.788 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 00:51:46.788 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 00:51:46.789 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 00:51:46.789 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:46.790 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:57.531 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 02/06/2025 00:51:57.531 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 00:51:57.532 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 00:51:57.532 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 00:51:57.532 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 00:51:57.532 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:51:57.532 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:51:57.532 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 00:51:57.533 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 00:51:57.534 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 00:52:11.435 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 02/06/2025 00:52:11.435 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 00:52:11.435 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/06/2025 00:52:11.436 7f37afee1048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 02/06/2025 00:52:11.436 7f37afee1048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 00:52:11.436 7f37afee1048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:52:11.436 7f37afee1048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:52:11.436 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 00:52:11.437 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 00:52:31.008 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:52:34.503 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:52:39.526 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14218088, partition_space_available_1k=66872396, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2050780, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=161756, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50996, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8609964, free=340416, buffers=665112, cached=6560144, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.14, five_min_avg=4.99, fifteen_min_avg=2.85, cpu_share=19.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:53:31.020 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:53:33.346 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:53:38.372 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14293776, partition_space_available_1k=66796708, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2055068, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152896, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51000, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8491692, free=455920, buffers=665464, cached=6562560, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.9, five_min_avg=5.0, fifteen_min_avg=2.98, cpu_share=17.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:54:31.107 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:54:33.854 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:54:35.377 7f37afee1048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:54:38.886 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14769404, partition_space_available_1k=66321080, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2049740, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=124852, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51036, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8550508, free=356720, buffers=670552, cached=6597856, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.15, five_min_avg=5.12, fifteen_min_avg=3.14, cpu_share=15.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:55:13.985 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:55:31.046 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:55:33.315 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:55:38.336 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14783780, partition_space_available_1k=66306704, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2091244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=127124, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51072, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8541652, free=365428, buffers=670668, cached=6597888, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.5, five_min_avg=5.4, fifteen_min_avg=3.36, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:56:31.021 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:56:33.312 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:56:38.340 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14787536, partition_space_available_1k=66302948, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2101372, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132524, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51116, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8560344, free=362968, buffers=670888, cached=6581436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.77, five_min_avg=5.13, fifteen_min_avg=3.4, cpu_share=14.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:57:31.075 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:57:33.386 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:57:38.409 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14787652, partition_space_available_1k=66302832, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2096380, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138464, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51156, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8580884, free=357360, buffers=670936, cached=6566456, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.7, five_min_avg=4.99, fifteen_min_avg=3.45, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:58:31.052 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:58:33.395 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:58:38.417 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14807204, partition_space_available_1k=66283280, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2106532, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143768, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51156, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8604876, free=334780, buffers=670612, cached=6565368, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.06, five_min_avg=4.75, fifteen_min_avg=3.47, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 00:59:31.054 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 00:59:33.275 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 00:59:38.299 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14816860, partition_space_available_1k=66273624, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2131596, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148960, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51160, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8627700, free=352032, buffers=670508, cached=6525396, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=4.45, fifteen_min_avg=3.45, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:00:31.038 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:00:33.469 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:00:38.492 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14820996, partition_space_available_1k=66269488, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2171736, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154428, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51164, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8683820, free=349524, buffers=670480, cached=6471812, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.61, five_min_avg=4.32, fifteen_min_avg=3.47, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:01:31.054 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:01:33.345 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:01:38.374 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14834920, partition_space_available_1k=66255564, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2183956, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=159688, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51192, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8705980, free=350536, buffers=670528, cached=6448592, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.21, five_min_avg=4.06, fifteen_min_avg=3.43, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:02:31.071 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:02:33.349 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:02:38.374 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14843332, partition_space_available_1k=66247152, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2213896, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=165212, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51212, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8737420, free=343360, buffers=670544, cached=6424312, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.99, five_min_avg=4.12, fifteen_min_avg=3.49, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:03:31.052 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:03:33.401 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:03:38.422 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14850132, partition_space_available_1k=66240352, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2223868, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170032, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51216, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8747232, free=354752, buffers=670592, cached=6403060, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.1, five_min_avg=4.13, fifteen_min_avg=3.53, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:04:31.046 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:04:33.341 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:04:38.363 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14866152, partition_space_available_1k=66224332, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2235464, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179368, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51224, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8793532, free=342652, buffers=670592, cached=6368860, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.07, five_min_avg=4.12, fifteen_min_avg=3.57, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:05:31.057 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:05:33.702 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:05:38.727 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14873340, partition_space_available_1k=66217144, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2240140, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184520, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51828, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8793100, free=350472, buffers=670596, cached=6361468, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=4.03, fifteen_min_avg=3.57, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:06:31.054 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:06:33.452 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:06:38.484 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14885644, partition_space_available_1k=66204840, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2249512, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189108, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51836, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8779556, free=406620, buffers=670592, cached=6318868, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.15, five_min_avg=3.82, fifteen_min_avg=3.53, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:07:31.117 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:07:33.601 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:07:38.625 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14894560, partition_space_available_1k=66195924, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2256256, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194484, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51832, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8810496, free=365312, buffers=670580, cached=6329248, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.02, five_min_avg=3.66, fifteen_min_avg=3.49, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:08:31.098 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:08:33.329 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:08:38.353 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14905772, partition_space_available_1k=66184712, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2270276, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=199604, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51844, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8834132, free=350092, buffers=670384, cached=6321028, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.66, five_min_avg=3.78, fifteen_min_avg=3.55, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:09:31.141 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:09:33.547 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:09:38.569 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14915508, partition_space_available_1k=66174976, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2281796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=205284, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51844, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8838832, free=345596, buffers=670392, cached=6320816, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=3.76, fifteen_min_avg=3.55, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:10:31.156 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:10:33.687 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:10:38.713 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14927440, partition_space_available_1k=66163044, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2283752, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=209208, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51828, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8858192, free=382428, buffers=670344, cached=6264672, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.3, five_min_avg=3.63, fifteen_min_avg=3.52, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:11:31.193 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:11:33.506 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:11:38.538 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14933712, partition_space_available_1k=66156772, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2287784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=210428, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51836, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8861604, free=367096, buffers=670348, cached=6276588, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.54, five_min_avg=3.92, fifteen_min_avg=3.63, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:12:31.206 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:12:33.492 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:12:38.513 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14948100, partition_space_available_1k=66142384, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2291912, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=216144, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51836, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8853736, free=372172, buffers=670344, cached=6279384, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.83, five_min_avg=3.84, fifteen_min_avg=3.62, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:13:31.238 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:13:33.540 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:13:38.562 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14938544, partition_space_available_1k=66151940, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2280492, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=221784, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51840, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8870536, free=356572, buffers=670364, cached=6278164, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=3.72, fifteen_min_avg=3.59, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:14:31.137 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:14:33.514 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:14:38.541 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14950708, partition_space_available_1k=66139776, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2284740, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=225560, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51844, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8858804, free=420676, buffers=670328, cached=6225828, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.19, five_min_avg=3.56, fifteen_min_avg=3.54, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:15:31.143 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:15:33.602 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:15:38.628 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14954332, partition_space_available_1k=66136152, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2283292, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=231216, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51844, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8887144, free=383700, buffers=670328, cached=6234464, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.11, five_min_avg=3.48, fifteen_min_avg=3.52, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:16:31.167 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:16:33.599 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:16:38.638 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14963340, partition_space_available_1k=66127144, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2282792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=236780, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51852, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8883460, free=378224, buffers=670328, cached=6243624, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.94, five_min_avg=3.37, fifteen_min_avg=3.48, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:17:31.133 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:17:33.471 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:17:38.493 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14977940, partition_space_available_1k=66112544, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2285412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=242588, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51848, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8893812, free=364660, buffers=670328, cached=6246836, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.46, fifteen_min_avg=3.5, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:18:31.183 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:18:33.905 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:18:38.927 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14984120, partition_space_available_1k=66106364, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2283904, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=246740, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51852, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8885924, free=419476, buffers=670292, cached=6199944, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.16, five_min_avg=3.35, fifteen_min_avg=3.46, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:19:31.162 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:19:33.517 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:19:38.542 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14991328, partition_space_available_1k=66099156, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2285416, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=252600, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51852, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8926528, free=366632, buffers=670292, cached=6212184, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.29, five_min_avg=3.4, fifteen_min_avg=3.47, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:20:31.140 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:20:33.476 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:20:38.499 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14999208, partition_space_available_1k=66091276, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2285560, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258204, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51860, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8924248, free=361020, buffers=670300, cached=6220068, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=3.46, fifteen_min_avg=3.49, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:21:31.144 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:21:33.644 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:21:38.681 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15010616, partition_space_available_1k=66079868, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2286876, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263300, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51864, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8908508, free=376896, buffers=670296, cached=6219936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.98, five_min_avg=3.34, fifteen_min_avg=3.44, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:22:31.160 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:22:34.219 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:22:39.245 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15013596, partition_space_available_1k=66076888, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2291304, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=267400, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51868, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8913084, free=412680, buffers=670264, cached=6179608, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.24, fifteen_min_avg=3.4, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:23:31.173 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:23:33.639 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:23:38.661 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15019636, partition_space_available_1k=66070848, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2287504, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=272948, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51880, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8936704, free=387600, buffers=670264, cached=6181068, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.06, five_min_avg=3.22, fifteen_min_avg=3.38, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:24:31.182 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:24:33.615 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:24:38.646 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15026744, partition_space_available_1k=66063740, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2290344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=278424, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8929064, free=382812, buffers=670272, cached=6193488, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.33, five_min_avg=3.28, fifteen_min_avg=3.39, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:25:31.232 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:25:33.643 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:25:38.663 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15038780, partition_space_available_1k=66051704, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2302616, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=283736, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51880, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8931412, free=380092, buffers=670268, cached=6193864, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.98, five_min_avg=3.21, fifteen_min_avg=3.36, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:26:31.195 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:26:34.169 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:26:39.220 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15040600, partition_space_available_1k=66049884, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2287908, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=288324, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51888, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8938292, free=384192, buffers=670256, cached=6182896, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=3.22, fifteen_min_avg=3.35, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:27:31.217 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:27:33.554 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:27:38.578 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15054704, partition_space_available_1k=66035780, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2291140, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=294280, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51888, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8971692, free=343276, buffers=670256, cached=6190412, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.22, fifteen_min_avg=3.35, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:28:31.244 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:28:33.732 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:28:38.756 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15062208, partition_space_available_1k=66028276, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2290844, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=299784, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51900, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8968164, free=335444, buffers=670260, cached=6201768, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=3.27, fifteen_min_avg=3.36, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:29:31.242 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:29:33.477 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:29:38.504 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15068816, partition_space_available_1k=66021668, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2290084, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=308036, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51900, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8942872, free=370952, buffers=670256, cached=6191556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.05, five_min_avg=3.19, fifteen_min_avg=3.33, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:30:31.237 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:30:34.138 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:30:39.165 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15077260, partition_space_available_1k=66013224, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2289636, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=310692, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51880, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8945076, free=419572, buffers=670176, cached=6140812, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=3.35, fifteen_min_avg=3.37, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:31:31.317 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:31:33.716 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:31:38.754 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15084312, partition_space_available_1k=66006172, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2289644, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=316760, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51876, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8963596, free=395308, buffers=670180, cached=6146552, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=3.36, fifteen_min_avg=3.37, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:32:31.245 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:32:33.433 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:32:38.456 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15090896, partition_space_available_1k=65999588, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2293112, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=322124, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8970632, free=376712, buffers=670180, cached=6158112, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=3.33, fifteen_min_avg=3.36, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:33:31.249 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:33:33.646 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:33:38.672 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15110716, partition_space_available_1k=65979768, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2292948, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=327736, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8945784, free=380788, buffers=670184, cached=6178880, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.88, five_min_avg=3.2, fifteen_min_avg=3.32, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:34:31.251 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:34:33.966 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:34:38.992 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15118400, partition_space_available_1k=65972084, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2293172, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=332512, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8917432, free=416976, buffers=670180, cached=6171048, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.0, five_min_avg=3.21, fifteen_min_avg=3.31, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:35:31.247 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:35:33.554 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:35:38.580 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15129124, partition_space_available_1k=65961360, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2293196, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=338664, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51892, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8948308, free=380168, buffers=670180, cached=6176980, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.19, five_min_avg=3.23, fifteen_min_avg=3.31, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:36:31.260 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:36:33.632 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:36:38.661 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15136336, partition_space_available_1k=65954148, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2294144, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=344120, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51888, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8954284, free=366892, buffers=670180, cached=6184280, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.23, fifteen_min_avg=3.31, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:37:31.294 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:37:33.644 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:37:38.669 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15144856, partition_space_available_1k=65945628, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2302672, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=349296, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51892, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8928256, free=386444, buffers=670176, cached=6190760, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=3.36, fifteen_min_avg=3.35, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:38:31.265 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:38:34.266 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:38:39.304 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15152048, partition_space_available_1k=65938436, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2294352, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=353132, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51888, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8958976, free=384348, buffers=670152, cached=6162160, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=3.35, fifteen_min_avg=3.34, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:39:31.243 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:39:33.528 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:39:38.551 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15161244, partition_space_available_1k=65929240, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2296000, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=358636, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51896, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8961160, free=372640, buffers=670156, cached=6171680, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.04, five_min_avg=3.28, fifteen_min_avg=3.32, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:40:31.293 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:40:33.625 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:40:38.652 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15167960, partition_space_available_1k=65922524, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2306016, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=364532, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51900, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8950396, free=377604, buffers=670156, cached=6177480, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.3, five_min_avg=3.33, fifteen_min_avg=3.34, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:41:31.359 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:41:33.705 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:41:38.739 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15179484, partition_space_available_1k=65911000, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2301212, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=369408, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51908, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8963260, free=368736, buffers=670156, cached=6173484, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.04, five_min_avg=3.23, fifteen_min_avg=3.3, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:42:31.268 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:42:33.980 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:42:39.005 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15184652, partition_space_available_1k=65905832, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2296460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=372080, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8941676, free=435084, buffers=670100, cached=6128776, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.88, five_min_avg=3.39, fifteen_min_avg=3.35, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:43:31.287 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:43:33.612 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:43:38.634 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15187548, partition_space_available_1k=65902936, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2296576, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=378104, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8963404, free=400196, buffers=670100, cached=6141936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.65, five_min_avg=3.4, fifteen_min_avg=3.36, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:44:31.310 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:44:33.777 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:44:38.801 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15199692, partition_space_available_1k=65890792, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2297928, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=383608, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51912, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8953168, free=403256, buffers=670100, cached=6149112, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.35, five_min_avg=3.36, fifteen_min_avg=3.35, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:45:31.314 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:45:33.645 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:45:38.668 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15202376, partition_space_available_1k=65888108, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2295436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=389172, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51928, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8942620, free=401272, buffers=670104, cached=6161640, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.41, five_min_avg=3.38, fifteen_min_avg=3.36, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:46:31.307 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:46:33.874 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:46:38.906 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15214948, partition_space_available_1k=65875536, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2295248, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=392188, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51904, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8946764, free=432624, buffers=670060, cached=6126188, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=3.37, fifteen_min_avg=3.35, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:47:31.334 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:47:33.622 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:47:38.646 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15225156, partition_space_available_1k=65865328, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2275976, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=398212, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51916, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8961128, free=407492, buffers=670080, cached=6136936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.32, fifteen_min_avg=3.34, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:48:31.348 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:48:33.686 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:48:38.715 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15228428, partition_space_available_1k=65862056, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2273104, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=403812, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51916, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8940612, free=419784, buffers=670080, cached=6145160, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=3.39, fifteen_min_avg=3.36, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:49:31.428 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:49:34.392 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:49:39.416 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15406280, partition_space_available_1k=65684204, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2270324, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=394276, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51804, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8940512, free=292576, buffers=678484, cached=6264064, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.44, five_min_avg=3.39, fifteen_min_avg=3.37, cpu_share=15.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:50:31.489 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:50:35.089 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:50:40.117 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15596456, partition_space_available_1k=65494028, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2267796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=374732, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51684, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9221972, free=330812, buffers=680092, cached=5942760, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.08, five_min_avg=3.9, fifteen_min_avg=3.54, cpu_share=19.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:51:31.659 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:51:37.626 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:51:42.663 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16517504, partition_space_available_1k=64572980, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2267408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=347256, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51276, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9415532, free=292292, buffers=694588, cached=5773224, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.48, five_min_avg=4.64, fifteen_min_avg=3.81, cpu_share=17.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:52:31.553 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:52:36.587 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:52:41.620 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17373784, partition_space_available_1k=63716700, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2270696, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=324164, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50872, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9460984, free=287564, buffers=744768, cached=5682320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=12.31, five_min_avg=6.39, fifteen_min_avg=4.46, cpu_share=21.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:53:29.969 7f37af18b348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:29.970 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/06/2025 01:53:29.981 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:29.983 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/06/2025 01:53:29.983 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:29.984 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:29.984 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:29.984 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:29.984 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:29.993 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:29.993 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:29.993 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.042 7f37af18b348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:30.042 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:30.043 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:30.043 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.043 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:30.043 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.043 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.043 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 2 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:30.044 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:31.362 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:53:31.776 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:31.776 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 18032128 disk_space_available_1k = 63058356 config_db_size_1k = 3124 >>, ] >> 02/06/2025 01:53:31.776 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:33.415 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:53:34.044 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:34.047 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:34.048 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:34.048 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:34.048 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:34.048 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:34.048 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:34.048 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:34.049 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:34.049 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:34.049 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:34.051 7f37af18b348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:34.051 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:34.051 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:34.051 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:34.051 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:34.052 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:34.052 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:34.052 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:34.052 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:38.052 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:38.054 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:38.055 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:38.055 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:38.055 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:38.440 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18028972, partition_space_available_1k=63061512, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2264816, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=326808, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50788, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8545544, free=1241940, buffers=731792, cached=5656360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.23, five_min_avg=6.41, fifteen_min_avg=4.59, cpu_share=18.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:53:38.440 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:42.801 7f37af18b348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:42.802 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:42.802 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:42.802 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.803 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:42.803 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.803 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.803 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:42.803 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.804 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:46.807 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:46.807 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:46.808 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:46.808 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:46.810 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:46.810 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:46.811 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:46.811 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.812 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:46.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.812 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:46.813 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.809 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:50.811 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:50.811 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:50.811 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:50.811 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:50.811 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:50.811 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:50.812 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.814 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:54.815 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:54.815 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:54.815 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:54.816 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:54.816 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:54.816 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:54.816 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:54.816 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.817 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:54.817 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.817 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.817 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:54.817 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.816 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:58.817 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:58.817 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:58.817 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:58.817 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.818 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.819 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:53:58.819 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.820 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:02.821 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:02.821 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:02.821 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.822 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.823 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:02.823 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.824 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:06.825 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:06.825 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:06.825 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:06.825 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:06.825 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:06.825 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:06.826 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.827 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:10.829 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:10.829 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:10.829 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:10.829 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:10.829 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:10.829 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:10.830 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.830 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:14.831 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:14.831 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:14.831 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:14.832 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.833 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.833 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:14.833 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.833 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:18.834 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:18.835 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:18.835 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:18.835 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:18.835 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:18.835 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:18.835 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:18.836 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.836 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:18.836 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.836 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.837 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:18.837 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.836 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:22.837 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:22.837 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.838 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:22.839 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.839 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.839 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:22.839 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.840 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:26.842 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:26.842 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:26.842 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:26.843 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:26.843 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:26.843 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:26.844 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:26.844 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.844 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:26.844 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.845 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.845 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:26.845 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.843 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:30.844 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:30.845 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:30.845 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:30.845 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:30.845 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:30.845 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:30.846 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:31.303 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:54:31.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:31.652 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 17926432 disk_space_available_1k = 63164052 config_db_size_1k = 3124 >>, ] >> 02/06/2025 01:54:31.652 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:33.240 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:54:34.846 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:34.847 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:34.847 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:34.848 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:34.848 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:34.848 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:34.848 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:34.848 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:34.848 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.849 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:34.849 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.849 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.849 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:34.849 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.277 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17926304, partition_space_available_1k=63164180, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2264584, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=327996, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50872, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7528200, free=2287564, buffers=733784, cached=5626088, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.4, five_min_avg=5.84, fifteen_min_avg=4.5, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:54:38.277 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.277 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17926304 partition_space_available_1k = 63164180 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2264584 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 327996 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 50872 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7528200 free = 2287564 buffers = 733784 cached = 5626088 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.4 five_min_avg = 5.84 fifteen_min_avg = 4.5 cpu_share = 9.25 node_type = config-database-node >> >> 02/06/2025 01:54:38.277 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.848 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:38.850 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:38.850 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:38.850 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:38.850 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:38.850 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.851 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:38.852 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.852 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:42.853 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:42.853 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:42.853 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:42.853 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.854 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.855 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:42.855 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.855 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:46.857 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:46.857 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:46.857 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:46.857 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:46.857 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:46.857 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:46.858 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.860 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:50.861 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:50.861 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:50.862 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:50.862 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:50.862 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:50.863 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:50.863 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:50.864 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.864 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:50.865 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.865 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.865 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:50.865 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.864 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:54.865 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:54.865 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:54.866 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:54.866 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:54.866 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:54.866 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:54.867 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:54.867 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.867 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:54.867 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.867 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.867 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:54.868 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.868 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:58.870 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:58.870 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:58.870 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:58.871 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:58.871 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:58.871 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:58.871 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:58.872 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.872 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:58.872 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.872 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.872 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:54:58.872 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.872 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:02.875 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:02.875 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:02.875 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:02.875 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:02.875 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:02.876 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:02.876 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:02.876 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.876 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:02.876 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.877 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.877 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:55:02.877 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.878 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:06.879 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:06.880 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:06.880 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:06.882 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:06.882 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:06.882 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:06.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:06.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.883 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:06.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.884 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:55:06.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.880 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:10.882 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:10.882 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:10.882 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:10.882 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:10.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:10.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:10.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:10.883 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.884 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:10.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.885 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:55:10.886 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.882 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:14.884 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:14.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:14.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:14.884 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:14.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:14.884 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:55:14.885 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.886 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:18.887 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:18.887 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:18.887 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:18.887 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.889 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:55:18.889 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.888 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:22.889 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:22.889 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:22.889 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:22.890 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:22.890 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:22.890 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:22.890 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:22.890 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.891 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:22.891 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.891 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.891 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1738802790290771 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 02/06/2025 01:55:22.891 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:26.892 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:26.895 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:55:26.895 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:26.895 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:26.896 7f37afee1348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:26.896 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:26.896 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:55:26.897 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:55:26.897 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:55:26.897 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:55:26.898 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:55:28.609 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 02/06/2025 01:55:28.610 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:55:28.610 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:55:28.611 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:55:28.611 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 01:55:28.612 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:55:28.612 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:55:28.613 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:55:28.614 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:55:28.626 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:55:31.451 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:55:38.154 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:55:38.602 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001441 02/06/2025 01:55:38.603 7f37af18b348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:55:38.604 7f37afee1348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/06/2025 01:55:38.604 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 02/06/2025 01:55:38.604 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 01:55:38.604 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:55:38.604 7f37afee1348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:55:38.605 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:55:38.605 7f37afee1048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:55:44.244 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18063256, partition_space_available_1k=63027228, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2265676, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=328040, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50956, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9024784, free=592544, buffers=741396, cached=5816912, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=9.14, five_min_avg=6.52, fifteen_min_avg=4.81, cpu_share=20.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:56:31.330 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:56:33.362 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:56:38.881 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18516824, partition_space_available_1k=62573660, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2263908, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=316212, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50864, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9057332, free=527576, buffers=740808, cached=5849920, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.24, five_min_avg=6.16, fifteen_min_avg=4.79, cpu_share=14.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:57:31.368 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:57:33.605 7f37b583bdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:57:38.882 7f37b583bdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18523540, partition_space_available_1k=62566944, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2266592, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=316220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50864, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9093352, free=484352, buffers=740812, cached=5857120, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.09, five_min_avg=5.5, fifteen_min_avg=4.64, cpu_share=9.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:58:17.230 7f37af18b148 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_process_state:zookeeper,group:zookeeper,state:PROCESS_STATE_EXITED 02/06/2025 01:58:17.230 7f37af18b148 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738807097230370, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1738802787000000', last_stop_time='', last_exit_time='1738807097230306', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738802787000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 01:58:44.191 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 01:58:44.191 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 01:58:44.191 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 01:58:44.200 7fb1dd132a20 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 02/06/2025 01:58:44.201 7fb1dd132a20 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 02/06/2025 01:58:44.204 7fb1dd136390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 01:58:44.205 7fb1dd136390 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738807124206053 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, 10.0.0.38:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 01:58:44.206 7fb1dd136390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.540 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 107563338465910168268255179396333207922469629543128463691982418335639062608953, 'start': '1738807120000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 33353924765468839689996568054027126947693467400867404952649245372826449198925, 'start': '1738807120000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 45509120408902478421124319597327991155111574013530218564710158867864818230955, 'start': '1738807121000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 01:58:44.697 7fb1dc9b1390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.698 7fb1dc9b1390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 02/06/2025 01:58:44.698 7fb1dc9b1390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.699 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738807124699047, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 01:58:44.699 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738807124699386, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807120000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807120000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807121000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 01:58:44.701 7fb1e5c3ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.702 7fb1e5c3ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:58:44.702 7fb1e5c3ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.702 7fb1e5c3ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:44.702 7fb1e5c3ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807120000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807120000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807121000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 02/06/2025 01:58:44.702 7fb1e5c3ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:45.419 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:58:46.203 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 01:58:46.204 7fb1dd2b3588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:46.205 7fb1dd2b3588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. database_usage = [ << disk_space_used_1k = 19409308 disk_space_available_1k = 61681176 config_db_size_1k = 4032 >>, ] >> 02/06/2025 01:58:46.205 7fb1dd2b3588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:47.214 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 01:58:47.214 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738807127214602, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 01:58:47.216 7fb1dc98ef60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:47.216 7fb1dc98ef60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:58:47.216 7fb1dc98ef60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:58:48.206 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:58:48.208 7fb1d7da1588 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:58:48.209 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:58:48.209 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:58:48.209 7fb1d7da1588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:58:48.218 7fb1d7da1240 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 01:58:48.219 7fb1d7da1240 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:58:48.219 7fb1d7da18d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:58:48.219 7fb1d7da18d0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:58:48.220 7fb1d7da18d0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/06/2025 01:58:48.220 7fb1d7da18d0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:58:48.221 7fb1d7da18d0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:58:48.221 7fb1d7da18d0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:58:48.222 7fb1dd20ef60 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:58:48.224 7fb1dd20ef60 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:58:50.287 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:58:52.429 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 02/06/2025 01:58:53.752 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 02/06/2025 01:58:55.295 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 02/06/2025 01:58:56.517 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 02/06/2025 01:58:58.709 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 02/06/2025 01:59:00.583 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 02/06/2025 01:59:05.617 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738807145616709, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807120000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807120000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807121000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 01:59:05.617 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19693388, partition_space_available_1k=61397096, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1245264, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=123372, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53080, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8177520, free=838680, buffers=717420, cached=6442016, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.27, five_min_avg=5.33, fifteen_min_avg=4.65, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 01:59:45.024 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 01:59:46.264 7fb1df9e06c8 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738807186263962, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.23:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/06/2025 01:59:47.609 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 01:59:52.969 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19564476, partition_space_available_1k=61526008, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1702144, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=126840, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53096, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8345716, free=677704, buffers=719592, cached=6432624, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.3, five_min_avg=5.12, fifteen_min_avg=4.62, cpu_share=15.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:00:45.065 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:00:47.585 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:00:53.268 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19572164, partition_space_available_1k=61518320, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1706468, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144612, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53096, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8381320, free=637208, buffers=719616, cached=6437492, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=4.73, fifteen_min_avg=4.51, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:01:45.077 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:01:47.094 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:01:52.261 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19574016, partition_space_available_1k=61516468, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1733452, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=169136, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53100, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8574408, free=429988, buffers=719664, cached=6451576, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.19, five_min_avg=4.44, fifteen_min_avg=4.42, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:02:45.086 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:02:47.942 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:02:53.015 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19580044, partition_space_available_1k=61510440, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1733220, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=174092, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53108, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8555032, free=432360, buffers=719948, cached=6468296, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=4.37, fifteen_min_avg=4.39, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:03:45.092 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:03:48.011 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:03:54.076 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19625468, partition_space_available_1k=61465016, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1736920, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=174628, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53152, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8588788, free=601608, buffers=709676, cached=6275564, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.33, five_min_avg=4.93, fifteen_min_avg=4.58, cpu_share=17.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:04:45.092 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:04:47.952 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:04:53.270 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19633704, partition_space_available_1k=61456780, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1743800, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=175036, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53120, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8359300, free=820172, buffers=717608, cached=6278556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.45, five_min_avg=4.6, fifteen_min_avg=4.49, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:05:45.117 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:05:47.504 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:05:53.282 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19638936, partition_space_available_1k=61451548, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1745028, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=175528, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53136, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8633268, free=526152, buffers=718612, cached=6297604, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=4.37, fifteen_min_avg=4.41, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:06:45.127 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:06:47.494 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:06:53.032 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19639564, partition_space_available_1k=61450920, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1749132, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=176740, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53136, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8613120, free=524236, buffers=722756, cached=6315524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.39, five_min_avg=4.16, fifteen_min_avg=4.34, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:07:45.169 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:07:47.613 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:07:53.270 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19644688, partition_space_available_1k=61445796, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1748852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=178432, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53136, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8644120, free=486660, buffers=722832, cached=6322024, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.04, five_min_avg=3.86, fifteen_min_avg=4.22, cpu_share=15.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:08:45.118 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:08:48.063 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:08:53.285 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19650016, partition_space_available_1k=61440468, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1752176, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=178968, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53136, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8751576, free=403604, buffers=720168, cached=6300288, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=3.62, fifteen_min_avg=4.11, cpu_share=14.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:09:45.110 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:09:47.601 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:09:53.287 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19672392, partition_space_available_1k=61418092, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1754536, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179452, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53124, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8762076, free=365092, buffers=720124, cached=6328344, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.09, five_min_avg=3.79, fifteen_min_avg=4.14, cpu_share=17.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:10:45.129 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:10:47.576 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:10:53.057 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19694488, partition_space_available_1k=61395996, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1753136, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179988, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53088, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8762912, free=400084, buffers=719108, cached=6293532, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.59, fifteen_min_avg=4.04, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:11:45.129 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:11:47.378 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:11:53.321 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19692984, partition_space_available_1k=61397500, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1754132, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=180460, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53088, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8621252, free=536492, buffers=719204, cached=6298688, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.58, five_min_avg=3.34, fifteen_min_avg=3.93, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:12:45.153 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:12:47.632 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:12:53.049 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19698828, partition_space_available_1k=61391656, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1755848, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=181008, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53092, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8687828, free=459996, buffers=719520, cached=6308292, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.58, five_min_avg=3.21, fifteen_min_avg=3.85, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:13:45.142 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:13:47.549 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:13:53.093 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19703228, partition_space_available_1k=61387256, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1756364, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183912, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53088, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8668456, free=464924, buffers=719748, cached=6322508, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.27, fifteen_min_avg=3.82, cpu_share=12.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:14:07.264 7fb1dd2f3a20 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:14:31.099 7fb1dd5816d8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:14:45.350 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:14:46.319 7fb1d7445630 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:14:47.642 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:14:53.330 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20060216, partition_space_available_1k=61030268, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1759660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184416, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53132, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8585232, free=414380, buffers=719452, cached=6456572, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.17, five_min_avg=2.98, fifteen_min_avg=3.69, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:15:00.621 7fb1dc9a7390 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:29.568 7fb1d73154e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:45.117 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:15:47.259 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:15:52.349 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20048736, partition_space_available_1k=61041748, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1771832, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184956, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53136, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8513484, free=486600, buffers=719496, cached=6456056, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.53, five_min_avg=2.99, fifteen_min_avg=3.66, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:16:45.259 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:16:47.859 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:16:53.357 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21146228, partition_space_available_1k=59944256, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1769704, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185456, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8702920, free=350556, buffers=716124, cached=6406036, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.63, five_min_avg=2.88, fifteen_min_avg=3.57, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:17:45.146 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:17:47.510 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:17:53.105 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21171796, partition_space_available_1k=59918688, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1782560, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=193848, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52044, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8948136, free=412988, buffers=706256, cached=6108256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.16, five_min_avg=2.97, fifteen_min_avg=3.55, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:18:45.180 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:18:48.593 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:18:54.362 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21191080, partition_space_available_1k=59899404, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1791432, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=193932, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51964, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9101868, free=324576, buffers=702160, cached=6047032, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.32, five_min_avg=3.86, fifteen_min_avg=3.82, cpu_share=15.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:19:45.179 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:19:47.315 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:19:53.126 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21228532, partition_space_available_1k=59861952, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1797628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=193968, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9126272, free=362288, buffers=699360, cached=5987716, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.97, five_min_avg=3.87, fifteen_min_avg=3.82, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:20:45.220 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:20:48.566 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:20:54.182 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21239700, partition_space_available_1k=59850784, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1786344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=193980, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51860, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9163992, free=342316, buffers=698636, cached=5970692, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.73, five_min_avg=4.22, fifteen_min_avg=3.94, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:21:45.241 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:21:47.725 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:21:53.152 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21250512, partition_space_available_1k=59839972, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1804864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=193992, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51824, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9184100, free=371280, buffers=695936, cached=5924320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.7, five_min_avg=4.4, fifteen_min_avg=4.01, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:22:45.418 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:22:48.411 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:22:54.379 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21288460, partition_space_available_1k=59802024, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1787328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194004, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51804, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9175452, free=357264, buffers=694640, cached=5948280, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.62, five_min_avg=4.34, fifteen_min_avg=4.01, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:23:45.223 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:23:47.600 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:23:53.141 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21303764, partition_space_available_1k=59786720, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1786572, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194012, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51780, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=9009920, free=516928, buffers=692756, cached=5956032, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.11, five_min_avg=3.94, fifteen_min_avg=3.89, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:24:45.211 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/06/2025 02:24:47.327 7fb1df9e06c8 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/06/2025 02:24:52.396 7fb1df9e06c8 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21356000, partition_space_available_1k=59734484, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1800276, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194008, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51788, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8963620, free=495200, buffers=695272, cached=6021544, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.94, fifteen_min_avg=3.9, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/06/2025 02:25:08.622 7fb1df9a6198 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map