10/28/2024 01:53:59.839 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/28/2024 01:53:59.840 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/28/2024 01:53:59.840 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/28/2024 01:53:59.848 7f76a5ca2248 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 10/28/2024 01:53:59.849 7f76a5ca2248 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 10/28/2024 01:53:59.850 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/28/2024 01:53:59.851 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:53:59.851 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/28/2024 01:53:59.851 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/28/2024 01:53:59.851 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:53:59.852 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/28/2024 01:53:59.852 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:53:59.852 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:53:59.852 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/28/2024 01:53:59.852 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:53:59.981 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 67180487240055741848353246329267529666436788066846446381451505652366340045567, 'start': '1730080437000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 81890907436509191273553290476605520200905261240761648805802569621720925165293, 'start': '1730080437000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 77415315398615437387604314202643619696907130771211621039111504805756825254390, 'start': '1730080437000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/28/2024 01:54:00.176 7f76a61e5648 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.176 7f76a61e5648 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-28 01:06:43.619366", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6193-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 10/28/2024 01:54:00.176 7f76a61e5648 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.177 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080440177098, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:54:00.177 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080440177426, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730080437000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730080437000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730080437000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-28 01:06:43.619366", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6193-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:54:00.179 7f76a61e5648 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.179 7f76a61e5648 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/28/2024 01:54:00.179 7f76a61e5648 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.180 7f76a61e5648 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.180 7f76a61e5648 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730080437000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730080437000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730080437000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-28 01:06:43.619366", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6193-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/28/2024 01:54:00.180 7f76a61e5648 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.474 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 01:54:00.752 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/28/2024 01:54:00.756 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:00.756 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 database_usage = [ << disk_space_used_1k = 50984168 disk_space_available_1k = 30106316 config_db_size_1k = 4 >>, ] >> 10/28/2024 01:54:00.756 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:01.434 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 10/28/2024 01:54:01.435 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080441435287, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:54:01.437 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:01.437 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/28/2024 01:54:01.437 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:03.350 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 10/28/2024 01:54:03.851 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:03.852 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:03.852 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:03.852 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:03.852 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:03.852 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:03.853 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:03.853 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:03.853 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:03.853 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:03.853 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:03.853 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:03.854 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:03.854 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:04.946 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 10/28/2024 01:54:05.384 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:54:06.406 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 10/28/2024 01:54:07.469 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 10/28/2024 01:54:07.853 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:07.854 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:07.854 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:07.854 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:07.855 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:07.855 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:07.855 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:07.855 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:07.855 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:07.855 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:07.856 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:07.856 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:07.856 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:07.856 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:08.535 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 10/28/2024 01:54:09.688 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 10/28/2024 01:54:10.945 7f76ac2c8f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 10/28/2024 01:54:11.855 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:11.857 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:11.857 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:11.858 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:11.858 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:11.858 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:11.858 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:11.859 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:11.859 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:11.859 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:11.860 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:11.860 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:11.860 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:11.860 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:12.521 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:54:15.858 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:15.864 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:15.865 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:15.865 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:15.865 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:15.865 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:15.865 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:15.866 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:15.989 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080455989066, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730080437000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730080437000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1730080437000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-28 01:06:43.619366", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6193-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:54:15.999 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51032872, partition_space_available_1k=30057612, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1234280, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52844, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51036, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11570076, free=12005724, buffers=663036, cached=8426276, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.17, five_min_avg=4.46, fifteen_min_avg=2.9, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 01:54:15.999 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:15.999 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730080437000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730080437000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1730080437000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2024-10-28 01:06:43.619366", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-6193-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 10/28/2024 01:54:16.000 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:16.000 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:16.000 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51032872 partition_space_available_1k = 30057612 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1234280 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 52844 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51036 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 11570076 free = 12005724 buffers = 663036 cached = 8426276 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 7.17 five_min_avg = 4.46 fifteen_min_avg = 2.9 cpu_share = 0 node_type = config-database-node >> >> 10/28/2024 01:54:16.000 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:19.868 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:19.869 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:19.869 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:19.870 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:19.870 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:19.870 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:19.870 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:19.870 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:19.870 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:19.871 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:19.871 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:19.871 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:19.871 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:19.871 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:23.872 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:23.874 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:23.874 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:23.875 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:23.875 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:23.875 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:23.875 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:23.876 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:23.877 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:23.877 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:23.877 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:23.877 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:23.877 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:23.877 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:23.926 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:54:27.876 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:27.878 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/28/2024 01:54:27.878 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:27.878 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:27.878 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:27.879 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:27.880 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/28/2024 01:54:27.881 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/28/2024 01:54:27.881 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:27.881 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/28/2024 01:54:27.882 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:27.882 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:27.882 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-441-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1730080439851482 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/28/2024 01:54:27.882 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/28/2024 01:54:31.879 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/28/2024 01:54:31.886 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 10/28/2024 01:54:31.887 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/28/2024 01:54:31.887 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:31.887 7f76a5ca2448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-441-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/28/2024 01:54:31.887 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/28/2024 01:54:31.887 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/28/2024 01:54:31.887 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 10/28/2024 01:54:31.888 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/28/2024 01:54:31.888 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/28/2024 01:54:31.889 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/28/2024 01:54:31.898 7f76a61e5648 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001012 10/28/2024 01:54:31.899 7f76a61e5648 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/28/2024 01:54:31.899 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 10/28/2024 01:54:31.900 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 10/28/2024 01:54:31.900 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/28/2024 01:54:31.900 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:54:31.900 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/28/2024 01:54:31.901 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 10/28/2024 01:54:31.901 7f76a5ca2548 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/28/2024 01:54:31.902 7f76a5ca2548 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/28/2024 01:54:54.980 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:55:00.697 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 01:55:03.257 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080503257654, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.30:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:55:05.200 7f76a61e5648 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001433 10/28/2024 01:55:05.201 7f76a61e5648 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/28/2024 01:55:05.201 7f76a5ca2448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 10/28/2024 01:55:05.201 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 10/28/2024 01:55:05.201 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/28/2024 01:55:05.202 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:55:05.202 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/28/2024 01:55:05.202 7f76a5ca2548 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/28/2024 01:55:05.203 7f76a5ca2548 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/28/2024 01:55:09.998 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:55:10.201 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080510201243, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:55:10.201 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:CassandraStatusUVE] map 10/28/2024 01:55:15.531 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51490356, partition_space_available_1k=29600128, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=198384, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45452, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14723928, free=10017956, buffers=727640, cached=7195588, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=22.67, five_min_avg=8.94, fifteen_min_avg=4.51, cpu_share=10.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 01:55:33.540 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:55:40.252 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:55:44.945 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:56:00.506 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 01:56:09.504 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51507880, partition_space_available_1k=29582604, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1313628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=218808, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45540, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14631568, free=11275884, buffers=740040, cached=6017620, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=13.79, five_min_avg=8.81, fifteen_min_avg=4.72, cpu_share=6.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 01:56:19.895 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:56:26.901 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:57:00.505 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 01:57:09.512 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51553288, partition_space_available_1k=29537196, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1307532, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=218852, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45592, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14763100, free=10842676, buffers=741160, cached=6318176, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.62, five_min_avg=8.24, fifteen_min_avg=4.78, cpu_share=3.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 01:58:00.504 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 01:58:09.520 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51558672, partition_space_available_1k=29531812, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305092, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=218896, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45600, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14758276, free=10839556, buffers=741332, cached=6325948, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.65, five_min_avg=7.15, fifteen_min_avg=4.62, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 01:59:00.498 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 01:59:13.782 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730080753781875, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.30:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='NTP state unsynchronized. ')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 01:59:19.577 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51564276, partition_space_available_1k=29526208, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1306292, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=218960, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45908, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14779320, free=10787224, buffers=741636, cached=6356932, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=6.25, fifteen_min_avg=4.49, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 01:59:48.800 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 01:59:52.540 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:00:00.529 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:00:09.185 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51572088, partition_space_available_1k=29518396, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1308148, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=219128, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45660, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14773260, free=10713068, buffers=741636, cached=6437148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.65, five_min_avg=5.85, fifteen_min_avg=4.44, cpu_share=3.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:01:00.569 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:01:01.583 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:01:09.561 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51589172, partition_space_available_1k=29501312, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1308616, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=221876, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45664, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14888016, free=9317572, buffers=741732, cached=7717792, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.72, five_min_avg=6.54, fifteen_min_avg=4.78, cpu_share=6.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:01:19.874 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:02:00.537 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:02:09.559 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51600568, partition_space_available_1k=29489916, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1331008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=221960, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45708, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14844988, free=9269804, buffers=741760, cached=7808560, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.49, five_min_avg=6.21, fifteen_min_avg=4.78, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:03:00.536 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:03:09.573 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51606196, partition_space_available_1k=29484288, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1311860, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222024, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45712, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14829972, free=9278460, buffers=741848, cached=7814832, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=5.5, fifteen_min_avg=4.62, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:04:00.626 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:04:04.039 7f76ac2c8f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-441-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1730081044039293, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.30:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/28/2024 02:04:09.593 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51612524, partition_space_available_1k=29477960, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1311184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222068, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45716, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14826704, free=9274256, buffers=741856, cached=7822296, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.61, five_min_avg=5.13, fifteen_min_avg=4.55, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:04:31.421 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:04:35.085 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:05:00.633 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:05:10.590 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51626208, partition_space_available_1k=29464276, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1310512, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222120, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45720, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14888384, free=9186592, buffers=741936, cached=7848200, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.18, five_min_avg=5.45, fifteen_min_avg=4.69, cpu_share=4.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:06:00.562 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:06:09.593 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51632692, partition_space_available_1k=29457792, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1311372, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222152, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45732, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14851628, free=9212868, buffers=741948, cached=7858668, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.31, five_min_avg=5.04, fifteen_min_avg=4.6, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:06:17.794 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:06:22.574 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:07:00.576 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:07:09.588 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51638848, partition_space_available_1k=29451636, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1311492, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222204, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45700, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14850392, free=9206616, buffers=741968, cached=7866136, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.97, five_min_avg=4.58, fifteen_min_avg=4.47, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:08:00.604 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:08:09.606 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51644764, partition_space_available_1k=29445720, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321000, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222260, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45700, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14868892, free=9180524, buffers=742112, cached=7873584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=4.33, fifteen_min_avg=4.38, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:09:00.606 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:09:00.615 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:09:09.805 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:09:15.690 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51658244, partition_space_available_1k=29432240, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1310580, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222328, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45732, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14972880, free=9056932, buffers=742116, cached=7893184, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.49, five_min_avg=5.28, fifteen_min_avg=4.69, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:10:00.560 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:10:09.599 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51664800, partition_space_available_1k=29425684, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1312752, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222376, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45740, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14888072, free=9130192, buffers=742116, cached=7904732, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.69, five_min_avg=4.76, fifteen_min_avg=4.54, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:11:00.622 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:11:03.180 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:11:09.606 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51665868, partition_space_available_1k=29424616, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1313480, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222440, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45744, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14901108, free=9115492, buffers=742120, cached=7906392, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=4.63, fifteen_min_avg=4.51, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:12:00.565 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:12:09.601 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51671532, partition_space_available_1k=29418952, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1312980, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222504, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45748, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14916200, free=9093788, buffers=742120, cached=7913004, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.62, five_min_avg=4.35, fifteen_min_avg=4.42, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:12:14.435 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:13:00.566 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:13:09.606 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51667888, partition_space_available_1k=29422596, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1313176, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222536, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45748, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14889728, free=9121576, buffers=742224, cached=7911584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.74, five_min_avg=3.95, fifteen_min_avg=4.28, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:14:00.569 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:14:09.629 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51673684, partition_space_available_1k=29416800, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1313208, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222596, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45752, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14889880, free=9113564, buffers=742336, cached=7919332, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.57, five_min_avg=3.67, fifteen_min_avg=4.16, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:14:53.097 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:14:56.936 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:15:00.572 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:15:09.609 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51682036, partition_space_available_1k=29408448, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1314172, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222624, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45760, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14947432, free=9035828, buffers=742336, cached=7939516, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.29, five_min_avg=4.66, fifteen_min_avg=4.47, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:16:00.657 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:16:09.624 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51687796, partition_space_available_1k=29402688, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1313632, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222664, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46364, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15058420, free=8913608, buffers=742344, cached=7950740, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.61, five_min_avg=4.15, fifteen_min_avg=4.31, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:16:09.689 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:17:00.620 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:17:09.639 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51688984, partition_space_available_1k=29401500, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1314128, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222712, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46364, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14947040, free=9013332, buffers=743076, cached=7961664, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=4.16, fifteen_min_avg=4.3, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:17:38.613 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:18:00.661 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:18:09.662 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51690140, partition_space_available_1k=29400344, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1314572, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222752, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46372, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14946572, free=9011896, buffers=743092, cached=7963552, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.13, five_min_avg=4.49, fifteen_min_avg=4.4, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:19:00.612 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:19:09.711 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51695760, partition_space_available_1k=29394724, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1314644, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222780, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46368, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15202888, free=8741172, buffers=743092, cached=7977960, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=4.13, fifteen_min_avg=4.28, cpu_share=3.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:19:57.690 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:20:00.622 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:20:01.462 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:20:09.667 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51704940, partition_space_available_1k=29385544, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1316184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222824, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46376, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14975352, free=8957356, buffers=743100, cached=7989304, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.71, five_min_avg=4.79, fifteen_min_avg=4.51, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:20:37.439 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:21:00.658 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:21:09.670 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51710928, partition_space_available_1k=29379556, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1316704, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222852, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46384, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14966436, free=8958888, buffers=743108, cached=7996680, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=4.41, fifteen_min_avg=4.39, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:22:00.651 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:22:09.670 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51707028, partition_space_available_1k=29383456, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1316480, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222884, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46376, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14966436, free=8962124, buffers=743120, cached=7993432, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.68, five_min_avg=4.56, fifteen_min_avg=4.45, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:23:00.698 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:23:09.682 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51712652, partition_space_available_1k=29377832, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1316660, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222920, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46380, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15009332, free=8912612, buffers=743120, cached=8000048, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.83, five_min_avg=4.05, fifteen_min_avg=4.28, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:23:13.697 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:24:00.677 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:24:09.705 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51713732, partition_space_available_1k=29376752, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1316488, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=222960, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46384, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14976716, free=8943456, buffers=743124, cached=8001816, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.67, five_min_avg=3.84, fifteen_min_avg=4.2, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:24:58.601 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:25:00.689 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:25:02.537 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:25:09.692 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51727964, partition_space_available_1k=29362520, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=223004, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46388, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14999176, free=8997716, buffers=743124, cached=7925096, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.64, five_min_avg=4.41, fifteen_min_avg=4.38, cpu_share=4.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:25:26.645 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:26:00.669 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:26:09.697 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51724184, partition_space_available_1k=29366300, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229216, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46392, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15009840, free=8989724, buffers=743124, cached=7922424, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=4.18, fifteen_min_avg=4.3, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:27:00.658 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:27:09.699 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51729956, partition_space_available_1k=29360528, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1318236, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229216, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46396, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14984776, free=9008544, buffers=743124, cached=7928668, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=3.96, fifteen_min_avg=4.21, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:28:00.694 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:28:09.697 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51735568, partition_space_available_1k=29354916, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1318684, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46412, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14997504, free=8989648, buffers=743124, cached=7934836, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=3.76, fifteen_min_avg=4.13, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:28:53.708 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:29:00.677 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:29:09.716 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51743588, partition_space_available_1k=29346896, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46400, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15014908, free=8954036, buffers=743128, cached=7953040, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=7.35, five_min_avg=4.87, fifteen_min_avg=4.48, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:29:48.374 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:29:53.276 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:29:57.026 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:30:00.713 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:30:09.714 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51750708, partition_space_available_1k=29339776, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1320324, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229216, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46416, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15033432, free=8906712, buffers=744312, cached=7980656, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.67, five_min_avg=4.53, fifteen_min_avg=4.39, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:31:00.702 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:31:09.704 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51747168, partition_space_available_1k=29343316, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319528, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229224, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46408, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15007936, free=8934444, buffers=744320, cached=7978412, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.1, five_min_avg=4.09, fifteen_min_avg=4.24, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:32:00.711 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:32:09.708 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51752484, partition_space_available_1k=29338000, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319592, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46408, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15008588, free=8927928, buffers=744320, cached=7984276, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.78, fifteen_min_avg=4.12, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:33:00.705 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:33:09.711 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51760180, partition_space_available_1k=29330304, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1320092, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46416, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15033868, free=8885320, buffers=744320, cached=8001604, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.81, five_min_avg=4.23, fifteen_min_avg=4.26, cpu_share=4.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:34:00.708 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:34:02.884 7f76a5ca2448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:34:09.731 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51765972, partition_space_available_1k=29324512, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321208, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229264, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46416, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15070628, free=8841944, buffers=744320, cached=8008220, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.02, five_min_avg=3.82, fifteen_min_avg=4.12, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:34:36.102 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:34:39.869 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:35:00.696 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:35:09.710 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51768416, partition_space_available_1k=29322068, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229264, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46420, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15030428, free=8878196, buffers=744320, cached=8012168, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.62, five_min_avg=3.8, fifteen_min_avg=4.09, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:35:21.437 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/28/2024 02:36:00.712 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:36:09.734 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51769628, partition_space_available_1k=29320856, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321708, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229264, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46428, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15031148, free=8875336, buffers=744320, cached=8014308, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=3.59, fifteen_min_avg=4.0, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:37:00.778 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/28/2024 02:37:09.736 7f76ac2c8f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-441-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51775932, partition_space_available_1k=29314552, percentage_partition_space_used=64, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1321792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229260, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46428, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15093768, free=8792968, buffers=744320, cached=8034056, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.21, five_min_avg=4.29, fifteen_min_avg=4.2, cpu_share=4.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/28/2024 02:37:32.862 7f76a5ca2548 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map