09/04/2025 01:43:37.757 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/04/2025 01:43:37.757 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/04/2025 01:43:37.757 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/04/2025 01:43:37.768 7feeb5fa34a0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 09/04/2025 01:43:37.768 7feeb5fa34a0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 09/04/2025 01:43:37.768 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/04/2025 01:43:37.769 7feeb5fa3860 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:37.786 7feeb5fa3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1756950217786125 collector_name = collector_ip = collector_list = [ 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/04/2025 01:43:37.787 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.123 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 16454217483316611250836642062335249057499262376483490997261988303946434467864, 'start': '1756950213000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 100811894359140766023747300294245582935009527095541303668016158526997055722170, 'start': '1756950213000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 5743497001544783740366828901085815458071595913314634723356398406407214326446, 'start': '1756950214000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/04/2025 01:43:38.247 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.248 7feeb5fa3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-04 00:15:17.050102", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10759-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 09/04/2025 01:43:38.249 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.249 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1456-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1756950218249329, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/04/2025 01:43:38.250 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1456-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1756950218250820, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756950213000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756950213000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756950214000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-04 00:15:17.050102", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10759-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/04/2025 01:43:38.255 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.255 7feeb5fa3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/04/2025 01:43:38.257 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.257 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.257 7feeb5fa3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756950213000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756950213000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756950214000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-04 00:15:17.050102", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10759-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/04/2025 01:43:38.258 7feeb5fa3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:38.698 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:43:39.383 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 09/04/2025 01:43:39.385 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:39.385 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 database_usage = [ << disk_space_used_1k = 12364412 disk_space_available_1k = 68726072 config_db_size_1k = 4 >>, ] >> 09/04/2025 01:43:39.385 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:39.715 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/04/2025 01:43:39.715 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1456-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1756950219715885, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/04/2025 01:43:39.717 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:39.717 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/04/2025 01:43:39.717 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:41.244 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:43:41.775 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 01:43:41.776 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 01:43:41.776 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:41.777 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1756950217786125 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 01:43:41.778 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:42.085 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:43:44.252 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 09/04/2025 01:43:45.752 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 09/04/2025 01:43:45.777 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 01:43:45.778 7feeb6085cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/04/2025 01:43:45.779 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:45.780 7feeb6085cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/04/2025 01:43:45.780 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:45.780 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:45.780 7feeb6085cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1756950217786125 collector_name = collector_ip = 10.0.0.19:8086 collector_list = [ 10.0.0.19:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/04/2025 01:43:45.780 7feeb6085cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/04/2025 01:43:48.093 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 09/04/2025 01:43:49.001 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 09/04/2025 01:43:49.698 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 09/04/2025 01:43:49.779 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/04/2025 01:43:49.780 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1456-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.19:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/04/2025 01:43:49.781 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/04/2025 01:43:49.782 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/04/2025 01:43:49.788 7feeb6085e00 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001013 09/04/2025 01:43:49.789 7feeb6085e00 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/04/2025 01:43:49.789 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 09/04/2025 01:43:49.789 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 09/04/2025 01:43:49.789 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 09/04/2025 01:43:49.789 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:43:49.789 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/04/2025 01:43:49.789 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 09/04/2025 01:43:49.790 7feeb6085540 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/04/2025 01:43:49.790 7feeb6085540 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/04/2025 01:43:50.602 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 09/04/2025 01:43:56.493 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1456-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1756950236493422, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756950213000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756950213000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756950214000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-04 00:15:17.050102", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10759-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/04/2025 01:43:56.494 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12763460, partition_space_available_1k=68327024, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1277816, cpu_share=0.0, mem_res=1265732, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58432, cpu_share=0.0, mem_res=53844, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56704, cpu_share=0.0, mem_res=44916, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=3817828, free=177176, buffers=390884, cached=11798320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.09, five_min_avg=2.35, fifteen_min_avg=1.07, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:44:01.822 7feeb6085e00 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001434 09/04/2025 01:44:01.823 7feeb6085e00 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/04/2025 01:44:01.824 7feeb6d65040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 09/04/2025 01:44:01.825 7feeb6d65040 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 09/04/2025 01:44:01.825 7feeb6d65040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 09/04/2025 01:44:01.825 7feeb6d65040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:44:01.825 7feeb6d65040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/04/2025 01:44:01.826 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/04/2025 01:44:01.827 7feeb5fa3f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/04/2025 01:44:03.865 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:44:15.164 7feeb5fa3900 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:44:28.014 7feeb5fa3e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:44:38.447 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:44:38.859 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:44:38.860 7feeb8e391c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1456-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1756950278860010, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.19:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/04/2025 01:44:39.630 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:44:44.767 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13254628, partition_space_available_1k=67835856, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342792, cpu_share=0.38, mem_res=1327360, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101720, cpu_share=0.07, mem_res=96292, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55816, cpu_share=0.01, mem_res=45024, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7165208, free=283192, buffers=357764, cached=8378044, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.11, five_min_avg=3.52, fifteen_min_avg=1.54, cpu_share=19.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:45:38.472 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:45:38.872 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:45:39.694 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:45:44.788 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13263388, partition_space_available_1k=67827096, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340144, cpu_share=0.03, mem_res=1325248, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132148, cpu_share=0.01, mem_res=126708, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55816, cpu_share=0.01, mem_res=45144, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7196488, free=279644, buffers=357972, cached=8350104, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.09, five_min_avg=3.2, fifteen_min_avg=1.55, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:46:38.513 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:46:38.983 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:46:39.979 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:46:45.831 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13270536, partition_space_available_1k=67819948, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340180, cpu_share=0.03, mem_res=1325608, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132512, cpu_share=0.0, mem_res=127368, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55844, cpu_share=0.01, mem_res=45052, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7267004, free=224020, buffers=369368, cached=8323816, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=3.29, fifteen_min_avg=1.68, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:46:46.642 7feeb62b24a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:46:49.448 7feeb62b24a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:46:53.575 7feeb62b24a0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:46:57.896 7feeb62b27c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:47:02.379 7feeb62b27c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:47:08.001 7feeb62b27c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:47:12.073 7feeb5fa3c20 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:47:17.145 7feeb5fa3c20 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:47:38.530 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:47:38.938 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:47:39.732 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:47:44.808 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13300128, partition_space_available_1k=67790356, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346476, cpu_share=0.2, mem_res=1331344, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=126228, cpu_share=0.01, mem_res=120884, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55928, cpu_share=0.01, mem_res=45208, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7437144, free=264136, buffers=368824, cached=8114104, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.53, five_min_avg=4.75, fifteen_min_avg=2.28, cpu_share=14.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:48:38.506 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:48:38.951 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:48:39.720 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:48:44.851 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13307656, partition_space_available_1k=67782828, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346908, cpu_share=0.03, mem_res=1331884, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=126676, cpu_share=0.0, mem_res=121280, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55916, cpu_share=0.01, mem_res=45104, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7458124, free=229976, buffers=369048, cached=8127060, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.99, five_min_avg=4.14, fifteen_min_avg=2.22, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:49:38.528 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:49:38.906 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:49:39.653 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:49:44.833 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13315392, partition_space_available_1k=67775092, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348332, cpu_share=0.03, mem_res=1332588, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=127100, cpu_share=0.0, mem_res=121676, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56008, cpu_share=0.01, mem_res=45044, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7455644, free=222680, buffers=369176, cached=8136708, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.2, five_min_avg=3.61, fifteen_min_avg=2.16, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:50:38.550 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:50:38.959 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:50:39.760 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:50:44.842 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13322696, partition_space_available_1k=67767788, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348284, cpu_share=0.03, mem_res=1332332, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=127676, cpu_share=0.0, mem_res=122336, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55908, cpu_share=0.01, mem_res=45072, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7460900, free=262900, buffers=368788, cached=8091620, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.68, five_min_avg=3.21, fifteen_min_avg=2.11, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:51:38.531 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:51:38.933 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:51:39.699 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:51:44.856 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13331192, partition_space_available_1k=67759292, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348128, cpu_share=0.03, mem_res=1332336, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=129856, cpu_share=0.0, mem_res=124580, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55900, cpu_share=0.01, mem_res=45016, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7501744, free=288936, buffers=369304, cached=8024224, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=3.18, fifteen_min_avg=2.17, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:52:00.080 7feeb626f360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:52:34.277 7feeb626f360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:52:37.089 7feeb626f360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:52:38.544 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:52:39.340 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:52:40.843 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:52:42.011 7feeb62b2220 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:52:46.889 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13343176, partition_space_available_1k=67747308, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348340, cpu_share=0.03, mem_res=1332640, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=130272, cpu_share=0.0, mem_res=124844, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56080, cpu_share=0.01, mem_res=45180, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7500068, free=321644, buffers=369116, cached=7993380, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.17, fifteen_min_avg=2.23, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:53:38.554 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:53:38.930 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:53:39.715 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:53:44.882 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13350372, partition_space_available_1k=67740112, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348340, cpu_share=0.03, mem_res=1332644, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=130680, cpu_share=0.0, mem_res=125372, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56120, cpu_share=0.01, mem_res=45104, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7491276, free=317484, buffers=369660, cached=8005788, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.33, five_min_avg=2.96, fifteen_min_avg=2.21, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:54:38.542 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:54:38.933 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:54:39.685 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:54:44.870 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13357868, partition_space_available_1k=67732616, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348992, cpu_share=0.03, mem_res=1332716, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=131136, cpu_share=0.0, mem_res=125900, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56096, cpu_share=0.01, mem_res=45140, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7502364, free=296816, buffers=369668, cached=8015360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.45, five_min_avg=2.84, fifteen_min_avg=2.22, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:55:38.589 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:55:39.004 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:55:39.768 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:55:44.877 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13365084, partition_space_available_1k=67725400, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348724, cpu_share=0.03, mem_res=1332560, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=131568, cpu_share=0.0, mem_res=126164, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56060, cpu_share=0.01, mem_res=45312, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7495280, free=295532, buffers=369696, cached=8023700, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=3.04, fifteen_min_avg=2.33, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:56:38.570 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:56:38.997 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:56:39.793 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:56:44.891 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13367908, partition_space_available_1k=67722576, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348980, cpu_share=0.03, mem_res=1332796, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132000, cpu_share=0.0, mem_res=126560, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56072, cpu_share=0.01, mem_res=45120, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7505256, free=272908, buffers=370692, cached=8035352, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.17, five_min_avg=3.07, fifteen_min_avg=2.38, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:57:06.956 7feeb623fd60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:57:38.610 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:57:39.042 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:57:39.964 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:57:45.903 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13370580, partition_space_available_1k=67719904, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1349040, cpu_share=0.03, mem_res=1332624, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132524, cpu_share=0.0, mem_res=127220, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56108, cpu_share=0.01, mem_res=44888, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7508072, free=265140, buffers=370748, cached=8040248, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=2.93, fifteen_min_avg=2.38, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:58:17.752 7feeb6085ae0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:58:20.585 7feeb6085ae0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:58:24.839 7feeb6085ae0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 01:58:38.597 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:58:38.978 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:58:39.759 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:58:44.922 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13384500, partition_space_available_1k=67705984, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348856, cpu_share=0.03, mem_res=1332752, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132980, cpu_share=0.0, mem_res=127616, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56156, cpu_share=0.01, mem_res=45028, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7512424, free=252144, buffers=370808, cached=8048832, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.06, fifteen_min_avg=2.45, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 01:59:38.606 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 01:59:39.007 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 01:59:39.776 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 01:59:44.915 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13392076, partition_space_available_1k=67698408, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350244, cpu_share=0.04, mem_res=1333504, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=134824, cpu_share=0.01, mem_res=129332, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56228, cpu_share=0.01, mem_res=45036, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7521912, free=238848, buffers=370832, cached=8052616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.05, five_min_avg=2.72, fifteen_min_avg=2.37, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:00:38.620 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:00:39.003 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:00:39.756 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:00:44.927 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13394132, partition_space_available_1k=67696352, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350472, cpu_share=0.03, mem_res=1333532, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144412, cpu_share=0.01, mem_res=138968, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56252, cpu_share=0.01, mem_res=45012, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7546128, free=205316, buffers=370864, cached=8061900, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.02, five_min_avg=2.56, fifteen_min_avg=2.33, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:01:38.640 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:01:39.019 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:01:39.768 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:01:44.938 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13396920, partition_space_available_1k=67693564, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350476, cpu_share=0.03, mem_res=1333676, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144876, cpu_share=0.0, mem_res=139496, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56272, cpu_share=0.01, mem_res=44996, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7540024, free=266620, buffers=369764, cached=8007800, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.11, five_min_avg=2.51, fifteen_min_avg=2.33, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:02:23.746 7feeb62b2220 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:02:27.606 7feeb62b2220 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:02:30.493 7feeb60852c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:02:35.139 7feeb60852c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:02:38.627 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:02:39.031 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:02:39.824 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:02:44.939 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13403312, partition_space_available_1k=67687172, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350428, cpu_share=0.03, mem_res=1333752, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=136512, cpu_share=0.0, mem_res=131172, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56340, cpu_share=0.01, mem_res=45176, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7544520, free=267596, buffers=368092, cached=8004000, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.24, five_min_avg=2.68, fifteen_min_avg=2.4, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:03:38.663 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:03:39.063 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:03:39.899 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:03:44.969 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13405288, partition_space_available_1k=67685196, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350476, cpu_share=0.04, mem_res=1333764, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=136960, cpu_share=0.0, mem_res=131568, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56384, cpu_share=0.01, mem_res=44968, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7542932, free=265920, buffers=368104, cached=8007252, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.11, five_min_avg=2.45, fifteen_min_avg=2.34, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:04:38.637 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:04:39.019 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:04:39.777 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:04:44.950 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13407836, partition_space_available_1k=67682648, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351636, cpu_share=0.03, mem_res=1334204, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=137436, cpu_share=0.0, mem_res=132096, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56232, cpu_share=0.01, mem_res=44948, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7546760, free=257948, buffers=368144, cached=8011356, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.81, five_min_avg=2.27, fifteen_min_avg=2.28, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:05:38.651 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:05:39.100 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:05:39.895 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:05:44.962 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13414804, partition_space_available_1k=67675680, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351780, cpu_share=0.03, mem_res=1334544, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=137908, cpu_share=0.0, mem_res=132492, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56300, cpu_share=0.01, mem_res=44912, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7549480, free=247580, buffers=368164, cached=8018984, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.26, five_min_avg=2.03, fifteen_min_avg=2.19, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:06:38.654 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:06:39.103 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:06:39.963 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:06:45.975 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13417392, partition_space_available_1k=67673092, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351816, cpu_share=0.03, mem_res=1334480, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138376, cpu_share=0.0, mem_res=132888, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56384, cpu_share=0.01, mem_res=45112, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7615868, free=248148, buffers=367244, cached=7952948, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.55, five_min_avg=1.94, fifteen_min_avg=2.15, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:07:25.380 7feeb62b2720 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:07:38.678 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:07:39.100 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:07:39.918 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:07:44.970 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13420240, partition_space_available_1k=67670244, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351652, cpu_share=0.03, mem_res=1334396, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138836, cpu_share=0.0, mem_res=133284, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56488, cpu_share=0.01, mem_res=45196, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7566460, free=283312, buffers=367764, cached=7966672, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.62, five_min_avg=1.89, fifteen_min_avg=2.12, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:08:06.138 7feeb623fd60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:08:09.148 7feeb623fd60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:08:14.125 7feeb623fd60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:08:38.699 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:08:39.137 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:08:39.942 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:08:45.988 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13426724, partition_space_available_1k=67663760, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351840, cpu_share=0.03, mem_res=1334404, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=139300, cpu_share=0.0, mem_res=133944, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56332, cpu_share=0.01, mem_res=45028, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7576612, free=312460, buffers=367796, cached=7927340, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.35, five_min_avg=1.76, fifteen_min_avg=2.05, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:09:38.681 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:09:39.099 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:09:39.882 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:09:44.994 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13434200, partition_space_available_1k=67656284, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352900, cpu_share=0.03, mem_res=1335140, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=139772, cpu_share=0.0, mem_res=134472, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56312, cpu_share=0.01, mem_res=44960, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7579520, free=300640, buffers=367828, cached=7936220, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.28, five_min_avg=1.69, fifteen_min_avg=2.01, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:10:38.698 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:10:39.155 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:10:39.923 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:10:45.002 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13426596, partition_space_available_1k=67663888, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353044, cpu_share=0.03, mem_res=1335424, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=140240, cpu_share=0.0, mem_res=134736, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56196, cpu_share=0.01, mem_res=45112, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7587896, free=298864, buffers=367832, cached=7929616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.61, five_min_avg=1.69, fifteen_min_avg=1.99, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:11:38.707 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:11:39.101 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:11:39.844 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:11:44.990 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13434184, partition_space_available_1k=67656300, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353092, cpu_share=0.03, mem_res=1335320, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=140708, cpu_share=0.0, mem_res=135396, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56332, cpu_share=0.01, mem_res=44948, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7597868, free=272920, buffers=368532, cached=7944888, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.85, five_min_avg=1.75, fifteen_min_avg=1.99, cpu_share=9.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:12:01.422 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:12:38.691 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:12:39.065 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:12:39.848 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:12:45.007 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13436828, partition_space_available_1k=67653656, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353220, cpu_share=0.03, mem_res=1335452, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=141176, cpu_share=0.0, mem_res=135792, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56260, cpu_share=0.01, mem_res=44900, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7604068, free=262124, buffers=368696, cached=7949320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.94, five_min_avg=1.77, fifteen_min_avg=1.98, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:13:38.696 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:13:39.107 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:13:39.894 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:13:45.024 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13438784, partition_space_available_1k=67651700, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353072, cpu_share=0.03, mem_res=1335344, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=141644, cpu_share=0.0, mem_res=136188, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56316, cpu_share=0.01, mem_res=45012, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7541404, free=320524, buffers=368796, cached=7953484, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=1.82, fifteen_min_avg=1.99, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:13:51.530 7feeb6d655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:13:54.565 7feeb6d655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:13:59.100 7feeb6d655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:14:38.711 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:14:39.082 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:14:39.844 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:14:45.012 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13445040, partition_space_available_1k=67645444, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354476, cpu_share=0.03, mem_res=1335912, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142232, cpu_share=0.0, mem_res=136848, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56332, cpu_share=0.01, mem_res=45184, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7545620, free=307856, buffers=368836, cached=7961896, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.06, five_min_avg=1.88, fifteen_min_avg=2.0, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:15:38.687 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:15:39.090 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:15:39.809 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:15:45.010 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13447092, partition_space_available_1k=67643392, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354548, cpu_share=0.03, mem_res=1335968, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142572, cpu_share=0.0, mem_res=137112, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56440, cpu_share=0.01, mem_res=45124, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7546192, free=304128, buffers=368848, cached=7965040, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.96, five_min_avg=1.88, fifteen_min_avg=1.99, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:16:04.698 7feeb5fa3b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:16:38.700 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:16:39.097 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:16:39.854 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:16:45.026 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13449384, partition_space_available_1k=67641100, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354320, cpu_share=0.03, mem_res=1335968, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143052, cpu_share=0.0, mem_res=137640, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56524, cpu_share=0.01, mem_res=45088, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7551532, free=295228, buffers=368868, cached=7968580, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.66, five_min_avg=1.85, fifteen_min_avg=1.97, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:17:38.700 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:17:39.098 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:17:39.839 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:17:45.022 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13457412, partition_space_available_1k=67633072, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354360, cpu_share=0.03, mem_res=1336176, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143500, cpu_share=0.0, mem_res=138036, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56424, cpu_share=0.01, mem_res=45076, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7565732, free=268196, buffers=368920, cached=7981360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.08, five_min_avg=1.96, fifteen_min_avg=2.0, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:18:38.703 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:18:39.082 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:18:40.014 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:18:46.038 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13467416, partition_space_available_1k=67623068, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354268, cpu_share=0.03, mem_res=1336028, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143988, cpu_share=0.0, mem_res=138564, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56520, cpu_share=0.01, mem_res=44848, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7571680, free=296208, buffers=367780, cached=7948540, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.72, five_min_avg=1.88, fifteen_min_avg=1.97, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:18:50.699 7feeb626f360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:18:53.256 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:18:57.445 7feeb5fa3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:19:38.714 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:19:39.174 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:19:39.941 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:19:45.032 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13473348, partition_space_available_1k=67617136, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355764, cpu_share=0.03, mem_res=1337068, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144432, cpu_share=0.0, mem_res=139092, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56560, cpu_share=0.01, mem_res=44844, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7578696, free=276212, buffers=368632, cached=7960668, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=2.11, fifteen_min_avg=2.05, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:20:35.090 7feeb5fa3b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:20:38.736 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:20:39.157 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:20:39.972 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:20:45.037 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13476060, partition_space_available_1k=67614424, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355740, cpu_share=0.03, mem_res=1336836, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144908, cpu_share=0.0, mem_res=139488, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56576, cpu_share=0.01, mem_res=44932, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7581012, free=270108, buffers=368652, cached=7964436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.82, five_min_avg=1.95, fifteen_min_avg=1.99, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:21:38.723 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:21:39.136 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:21:39.990 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:21:45.041 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13478212, partition_space_available_1k=67612272, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355512, cpu_share=0.03, mem_res=1336740, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145384, cpu_share=0.0, mem_res=140016, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56596, cpu_share=0.01, mem_res=44964, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7584100, free=264312, buffers=368668, cached=7967128, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.82, five_min_avg=1.94, fifteen_min_avg=1.99, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:22:38.782 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:22:39.293 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:22:40.138 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:22:46.033 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13486144, partition_space_available_1k=67604340, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355788, cpu_share=0.03, mem_res=1336672, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145860, cpu_share=0.0, mem_res=140412, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56636, cpu_share=0.01, mem_res=45168, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7595464, free=265552, buffers=367792, cached=7955400, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.27, five_min_avg=2.07, fifteen_min_avg=2.03, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:23:37.032 7feeb62b2ae0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:23:38.861 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:23:39.671 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:23:40.767 7feeb5fa3ea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:23:41.165 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:23:45.713 7feeb62b2e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:23:47.085 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13485276, partition_space_available_1k=67605208, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355648, cpu_share=0.03, mem_res=1336888, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146460, cpu_share=0.0, mem_res=140940, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56656, cpu_share=0.01, mem_res=44924, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7625812, free=328288, buffers=365652, cached=7864456, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.88, five_min_avg=2.19, fifteen_min_avg=2.07, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:24:38.789 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:24:39.222 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:24:40.137 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:24:46.043 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13489928, partition_space_available_1k=67600556, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356736, cpu_share=0.03, mem_res=1337536, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146792, cpu_share=0.0, mem_res=141336, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56692, cpu_share=0.01, mem_res=45068, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7605996, free=340084, buffers=365692, cached=7872436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.48, five_min_avg=2.19, fifteen_min_avg=2.08, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:25:36.925 7feeb62b2cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:25:38.725 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:25:39.094 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:25:39.933 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:25:45.062 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13492572, partition_space_available_1k=67597912, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1357084, cpu_share=0.03, mem_res=1337616, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147256, cpu_share=0.0, mem_res=141864, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56620, cpu_share=0.01, mem_res=45044, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7613200, free=328724, buffers=365696, cached=7876588, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.96, five_min_avg=2.52, fifteen_min_avg=2.19, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:26:38.752 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:26:39.167 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:26:39.916 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:26:45.089 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13494832, partition_space_available_1k=67595652, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356800, cpu_share=0.03, mem_res=1337432, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147716, cpu_share=0.0, mem_res=142260, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56796, cpu_share=0.01, mem_res=45276, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7612444, free=326120, buffers=365724, cached=7879920, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.07, five_min_avg=2.5, fifteen_min_avg=2.2, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:27:38.752 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/04/2025 02:27:39.183 7feeb8e391c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/04/2025 02:27:40.025 7feeb8e391c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/04/2025 02:27:45.093 7feeb8e391c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1456-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13497712, partition_space_available_1k=67592772, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356992, cpu_share=0.03, mem_res=1337524, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148180, cpu_share=0.0, mem_res=142788, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56732, cpu_share=0.01, mem_res=45268, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7626428, free=295712, buffers=367328, cached=7894740, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.24, five_min_avg=2.39, fifteen_min_avg=2.18, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/04/2025 02:27:54.349 7feeb60850e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:27:57.041 7feeb60850e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:28:01.190 7feeb60850e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/04/2025 02:28:34.778 7feeb62b2f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map