02/10/2025 00:45:20.603 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
02/10/2025 00:45:20.603 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
02/10/2025 00:45:20.603 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
02/10/2025 00:45:20.609 7f5f3deb3648 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
02/10/2025 00:45:20.610 7f5f3deb3648 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
02/10/2025 00:45:20.610 7f5f3deb3e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
02/10/2025 00:45:20.611 7f5f3deb3e48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 00:45:20.611 7f5f3deb3e48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 00:45:20.611 7f5f3deb3e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
02/10/2025 00:45:20.612 7f5f3deb3e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.612 7f5f3deb3e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 00:45:20.612 7f5f3deb3e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.612 7f5f3deb3e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.612 7f5f3deb3e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip =   collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
02/10/2025 00:45:20.612 7f5f3deb3e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.656 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 28811216344393028925944468355843025010761566813416766431978609851851944804363, 'start': '1739148318000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 55869751347553544739022065612541377075995567379996604790143112705088710265685, 'start': '1739148318000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 11736851940611091465110674595036301562930910179945874258079779730074413523049, 'start': '1739148318000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
02/10/2025 00:45:20.787 7f5f3d196148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.787 7f5f3d196148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7  installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7  >>  
02/10/2025 00:45:20.787 7f5f3d196148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.788 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739148320788267, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 00:45:20.788 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739148320788578, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739148318000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739148318000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739148318000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 00:45:20.790 7f5f3d196148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.790 7f5f3d196148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 00:45:20.790 7f5f3d196148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.790 7f5f3d196148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.791 7f5f3d196148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739148318000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739148318000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739148318000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
02/10/2025 00:45:20.791 7f5f3d196148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:20.990 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:45:21.180 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 00:45:21.181 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:21.181 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 4675472  disk_space_available_1k = 76415012  config_db_size_1k = 4  >>,  ]  >>  
02/10/2025 00:45:21.181 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:21.533 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
02/10/2025 00:45:21.533 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739148321533860, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 00:45:21.535 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:21.535 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 00:45:21.535 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:22.798 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:45:24.612 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:24.614 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:24.614 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:24.614 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:24.615 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:24.616 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:24.616 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:24.616 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:24.986 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
02/10/2025 00:45:25.997 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
02/10/2025 00:45:26.787 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
02/10/2025 00:45:27.668 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
02/10/2025 00:45:28.495 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
02/10/2025 00:45:28.614 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:28.616 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:28.616 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:28.617 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:28.617 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:28.617 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:28.617 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:28.617 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:28.618 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:28.618 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:28.618 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:28.618 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:28.618 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:28.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:29.576 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
02/10/2025 00:45:32.617 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:32.618 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:32.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:32.620 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:32.620 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:32.620 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:34.602 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739148334601931, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739148318000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739148318000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739148318000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 00:45:34.602 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=4964904, partition_space_available_1k=76125580, percentage_partition_space_used=6, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1232144, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58312, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51052, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=1720720, free=9966128, buffers=100320, cached=4388460, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.09, five_min_avg=0.63, fifteen_min_avg=0.22, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:45:34.602 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:34.602 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739148318000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739148318000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739148318000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
02/10/2025 00:45:34.602 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:34.603 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:34.603 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 4964904  partition_space_available_1k = 76125580  percentage_partition_space_used = 6  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1232144  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 58312  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51052  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 1720720  free = 9966128  buffers = 100320  cached = 4388460  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.09  five_min_avg = 0.63  fifteen_min_avg = 0.22  cpu_share = 0  node_type = config-database-node  >>  >>  
02/10/2025 00:45:34.603 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:36.619 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:36.621 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:36.621 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:36.621 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:36.621 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:36.621 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:36.621 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:36.622 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:40.622 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:40.623 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:40.623 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:40.623 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:40.624 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:40.625 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:40.625 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:44.627 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:44.691 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:44.691 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:44.691 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:44.691 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:44.691 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:44.691 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:44.692 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:44.692 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:44.692 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:44.692 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:44.692 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:44.692 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:44.693 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:48.691 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:48.693 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:48.693 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:48.693 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:48.693 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:48.694 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:48.695 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:48.695 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:52.695 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:52.696 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:52.696 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:52.696 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:52.696 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:52.696 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:52.696 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:52.697 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:56.700 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:45:56.701 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:45:56.701 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:45:56.701 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:56.702 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:45:56.703 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:45:56.703 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:00.705 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:00.707 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:00.707 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:00.707 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:00.707 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:00.708 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:00.708 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:00.708 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:00.708 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:00.708 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:00.708 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:00.709 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:00.709 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:00.709 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:04.710 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:04.711 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:04.711 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:04.711 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:04.712 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:04.712 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:04.712 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:04.712 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:04.712 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:04.712 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:04.713 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:04.713 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:04.713 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:04.713 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:08.713 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:08.714 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:08.715 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:08.715 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:08.715 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:08.715 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:08.715 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:08.715 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:08.716 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:08.716 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:08.716 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:08.716 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:08.716 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:08.716 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:12.718 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:12.719 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:12.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:12.721 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:12.721 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:12.721 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:12.721 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:12.721 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:16.723 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:16.725 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:16.725 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:16.725 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:16.725 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:16.725 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:16.725 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:16.726 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:20.726 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:20.727 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:20.728 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:20.729 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:20.729 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:20.729 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:20.729 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:20.729 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:20.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:20.730 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:20.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:20.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:20.730 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:20.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:20.996 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:46:21.166 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:21.166 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 5950768  disk_space_available_1k = 75139716  config_db_size_1k = 1544  >>,  ]  >>  
02/10/2025 00:46:21.167 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:21.718 7f5f43845df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739148381718897, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.30:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 00:46:21.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:21.720 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:21.720 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:22.458 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:46:24.728 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:24.730 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:24.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:24.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:24.730 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:24.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:24.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:24.731 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:27.480 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=5950784, partition_space_available_1k=75139700, percentage_partition_space_used=7, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1331120, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58172, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51400, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=1823700, free=8811368, buffers=174704, cached=5365856, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.88, five_min_avg=0.82, fifteen_min_avg=0.31, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:46:27.481 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:27.481 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 5950784  partition_space_available_1k = 75139700  percentage_partition_space_used = 7  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1331120  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 58172  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51400  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 1823700  free = 8811368  buffers = 174704  cached = 5365856  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.88  five_min_avg = 0.82  fifteen_min_avg = 0.31  cpu_share = 8.0  node_type = config-database-node  >>  >>  
02/10/2025 00:46:27.481 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:28.730 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:28.732 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:28.732 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:28.732 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:28.732 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:28.732 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:28.732 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:28.733 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:32.736 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:32.737 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:32.737 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:32.737 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:32.738 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:32.739 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:36.738 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:36.739 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:36.739 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:36.739 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:36.740 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:36.741 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:36.741 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:40.741 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:40.742 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:40.743 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:40.744 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:40.744 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:40.744 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:40.744 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:40.744 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:44.746 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:44.747 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:44.747 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:44.747 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:44.748 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:44.748 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:44.748 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:44.748 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:44.748 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:44.748 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:44.749 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:44.749 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:44.749 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:44.749 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:48.750 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:48.752 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:48.753 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:48.753 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:48.753 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:48.753 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:48.753 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:48.753 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:52.752 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:52.754 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:52.754 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:52.754 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:52.754 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:52.754 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:52.754 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:52.755 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:56.755 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:46:56.756 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:46:56.756 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:46:56.757 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:56.758 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:46:56.758 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:46:56.758 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:00.760 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:00.762 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:00.762 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:00.762 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:00.762 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:00.762 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:00.762 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:00.763 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:04.763 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:04.764 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:04.764 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:04.764 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:04.764 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:04.764 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:04.764 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:04.765 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:08.765 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:08.767 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:08.767 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:08.767 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:08.767 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:08.767 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:08.767 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:08.768 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:08.768 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:08.768 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:08.768 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:08.768 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:08.769 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:08.769 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:12.770 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:12.772 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:12.772 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:12.772 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:12.772 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:12.772 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:12.772 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:12.773 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:12.773 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:12.773 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:12.773 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:12.773 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:12.773 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:12.774 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:16.774 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:16.775 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:16.775 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:16.775 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:16.776 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:16.777 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:16.777 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:16.777 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:20.778 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:20.779 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:20.779 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:20.779 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:20.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:20.781 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:20.781 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:21.064 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:47:21.257 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:21.257 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 6250896  disk_space_available_1k = 74839588  config_db_size_1k = 1768  >>,  ]  >>  
02/10/2025 00:47:21.257 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:23.021 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:47:24.780 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:24.782 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:24.782 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:24.782 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:24.782 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:24.782 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:24.783 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:28.045 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6251140, partition_space_available_1k=74839344, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1634952, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76352, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51668, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=2713696, free=7536876, buffers=207488, cached=5717568, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.43, five_min_avg=1.18, fifteen_min_avg=0.47, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:47:28.045 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:28.046 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6251140  partition_space_available_1k = 74839344  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1634952  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 76352  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51668  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 2713696  free = 7536876  buffers = 207488  cached = 5717568  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.43  five_min_avg = 1.18  fifteen_min_avg = 0.47  cpu_share = 9.0  node_type = config-database-node  >>  >>  
02/10/2025 00:47:28.046 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:28.782 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:28.783 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:28.784 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:28.785 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:28.785 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:28.785 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:28.785 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:28.785 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:32.786 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:32.787 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:32.788 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:32.788 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:32.788 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:32.788 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:32.788 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:32.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:36.788 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:36.789 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:36.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:36.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:36.789 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:36.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:36.789 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:36.790 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:40.790 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:40.791 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:40.791 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:40.791 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:40.791 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:40.792 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:44.794 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:44.795 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:44.795 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:44.795 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:44.796 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:48.798 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:48.799 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:48.799 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:48.799 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:48.799 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:48.799 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:48.800 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:48.801 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:52.800 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:52.801 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:52.801 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:52.801 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:52.801 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:52.802 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:52.803 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:56.802 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:47:56.803 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:47:56.803 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:47:56.803 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:47:56.804 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:47:56.805 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:00.807 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:00.808 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:00.808 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:00.808 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:00.808 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:00.809 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:04.812 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:04.813 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:04.814 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:04.814 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:04.814 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:04.814 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:04.814 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:04.814 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:04.815 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:04.815 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:04.815 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:04.815 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:04.815 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:04.815 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:08.814 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:08.816 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:08.816 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:08.816 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:08.816 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:08.816 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:08.816 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:08.817 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:12.817 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:12.818 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:12.818 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:12.818 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:12.818 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:12.818 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:12.818 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:12.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:16.818 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:16.819 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:16.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:16.819 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:16.820 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:16.821 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:16.821 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:20.821 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:20.825 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:20.826 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:20.826 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:20.826 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:20.826 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:20.826 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:20.827 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:20.968 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:48:21.145 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:21.145 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 6370076  disk_space_available_1k = 74720408  config_db_size_1k = 1728  >>,  ]  >>  
02/10/2025 00:48:21.145 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:22.384 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:48:24.827 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:24.828 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:24.828 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:24.829 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:24.830 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:24.830 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:24.830 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:24.830 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:27.405 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6369952, partition_space_available_1k=74720532, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1727108, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78396, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51792, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=2974416, free=7147120, buffers=210032, cached=5844060, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.39, five_min_avg=1.1, fifteen_min_avg=0.48, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:48:27.406 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:27.406 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6369952  partition_space_available_1k = 74720532  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1727108  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 78396  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51792  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 2974416  free = 7147120  buffers = 210032  cached = 5844060  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.39  five_min_avg = 1.1  fifteen_min_avg = 0.48  cpu_share = 3.5  node_type = config-database-node  >>  >>  
02/10/2025 00:48:27.406 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:28.829 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:28.830 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:28.830 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:28.830 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:28.831 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:28.832 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:28.832 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:32.834 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:32.835 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:32.835 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:32.835 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:32.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:32.837 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:32.837 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:36.836 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:36.837 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:36.837 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:36.837 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:36.837 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:36.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:40.838 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:40.839 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:40.840 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:40.840 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:40.840 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:40.840 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:40.840 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:40.841 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:44.843 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:44.844 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:44.844 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:44.844 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:44.844 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:44.845 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:44.845 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:44.845 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:44.845 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:44.845 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:44.845 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:44.846 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:44.846 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:44.846 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:48.846 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:48.847 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:48.847 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:48.847 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:48.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:48.849 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:48.849 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:52.848 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:52.849 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:52.849 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:52.849 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:52.850 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:52.851 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:52.851 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:56.853 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:48:56.854 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:48:56.854 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:56.855 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:48:56.856 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:56.856 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:48:56.856 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:48:56.856 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:00.858 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:00.859 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:00.859 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:00.859 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:00.859 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:00.859 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:00.859 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:00.860 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:04.860 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:04.862 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:04.862 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:04.862 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:04.862 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:04.862 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:04.862 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:04.863 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:08.863 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:08.865 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:08.865 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:08.865 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:08.865 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:08.865 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:08.865 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:08.866 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:12.865 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:12.867 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:12.868 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:12.868 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:12.868 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:12.868 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:12.868 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:12.868 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:16.867 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:16.869 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:16.869 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:16.869 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:16.869 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:16.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:16.871 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:16.871 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:20.870 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:20.871 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:20.871 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:20.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:20.873 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:20.873 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:20.873 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:20.979 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:49:21.183 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:21.183 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 6759556  disk_space_available_1k = 74330928  config_db_size_1k = 1728  >>,  ]  >>  
02/10/2025 00:49:21.183 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:22.425 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:49:24.872 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:24.874 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:24.874 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:24.874 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:24.874 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:24.874 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:24.875 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:24.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:27.445 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6759564, partition_space_available_1k=74330920, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1722344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78620, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51976, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=4344308, free=5300740, buffers=237588, cached=6292992, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.69, five_min_avg=0.95, fifteen_min_avg=0.47, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:49:27.446 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:27.446 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6759564  partition_space_available_1k = 74330920  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1722344  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 78620  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51976  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 4344308  free = 5300740  buffers = 237588  cached = 6292992  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 0.69  five_min_avg = 0.95  fifteen_min_avg = 0.47  cpu_share = 3.75  node_type = config-database-node  >>  >>  
02/10/2025 00:49:27.446 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:28.875 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:28.876 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:28.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:28.876 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:28.877 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:28.878 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:32.878 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:32.879 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:32.879 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:32.879 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:32.880 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:32.881 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:32.881 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:32.882 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:36.882 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:36.884 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:36.884 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:36.884 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:36.884 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:36.884 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:36.884 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:36.885 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:40.886 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:40.888 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:40.888 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:40.888 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:40.888 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:40.888 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:40.888 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:40.889 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:44.890 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:44.891 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:44.891 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:44.891 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:44.892 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:44.893 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:48.894 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:48.895 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:48.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:48.897 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:48.897 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:52.896 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:52.897 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:52.897 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:52.897 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:52.897 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:52.898 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:52.899 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:56.900 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:49:56.902 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:49:56.902 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:49:56.902 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:56.902 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:49:56.902 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:49:56.902 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:49:56.903 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:00.906 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:50:00.907 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:50:00.907 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:50:00.907 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:00.907 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:50:00.908 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:04.909 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:50:04.911 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:50:04.911 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:50:04.911 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:04.911 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:04.912 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:04.913 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:50:04.913 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:08.913 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:50:08.914 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:50:08.914 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:50:08.914 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:08.915 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:08.916 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:50:08.916 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:12.918 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:50:12.919 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 00:50:12.919 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:50:12.919 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:12.919 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:12.920 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739148320611775  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/10/2025 00:50:12.921 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 00:50:16.922 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 00:50:16.923 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 00:50:16.924 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 00:50:16.925 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:16.925 7f5f3deeb348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 00:50:16.925 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 00:50:16.926 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 00:50:16.926 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 00:50:16.926 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 00:50:16.928 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 00:50:16.928 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 00:50:21.013 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:50:21.300 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 00:50:22.946 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:50:27.976 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6940408, partition_space_available_1k=74150076, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1726232, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=86424, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52312, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=4649604, free=4650920, buffers=251848, cached=6623256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.5, five_min_avg=1.09, fifteen_min_avg=0.54, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:50:27.976 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 00:50:28.067 7f5f3d196148 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
02/10/2025 00:50:28.068 7f5f3d196148 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/10/2025 00:50:28.068 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
02/10/2025 00:50:28.069 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
02/10/2025 00:50:28.069 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 6> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 00:50:28.069 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 161> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 00:50:28.069 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 75> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 00:50:28.070 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
02/10/2025 00:50:28.070 7f5f3deeb048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/10/2025 00:50:28.071 7f5f3deeb048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/10/2025 00:50:40.320 7f5f3d196148 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439
02/10/2025 00:50:40.322 7f5f3d196148 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/10/2025 00:50:40.323 7f5f3deeb048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
02/10/2025 00:50:40.324 7f5f3deeb048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
02/10/2025 00:50:40.324 7f5f3deeb048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 6> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 00:50:40.324 7f5f3deeb048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 162> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 00:50:40.325 7f5f3deeb048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 76> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 00:50:40.325 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/10/2025 00:50:40.326 7f5f3deeb348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/10/2025 00:51:21.049 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:51:23.112 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:51:28.134 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7232960, partition_space_available_1k=73857524, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1729320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=159748, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52528, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5626668, free=3471744, buffers=262872, cached=6814344, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=1.68, fifteen_min_avg=0.78, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:52:21.073 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:52:23.203 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:52:28.228 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7298028, partition_space_available_1k=73792456, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1729340, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154280, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52568, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5623272, free=3416208, buffers=264244, cached=6871904, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.05, five_min_avg=1.92, fifteen_min_avg=0.93, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:53:04.522 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 164> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 00:53:21.060 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:53:23.020 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:53:28.044 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7756976, partition_space_available_1k=73333508, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1733840, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=178420, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52656, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5696712, free=2861940, buffers=271012, cached=7345964, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.29, five_min_avg=2.21, fifteen_min_avg=1.09, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:53:44.035 7f5f3deeb348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 165> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 00:54:21.075 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:54:22.964 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:54:27.993 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7753048, partition_space_available_1k=73337436, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1793552, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=202248, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52672, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5772868, free=2773524, buffers=272944, cached=7356292, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.02, five_min_avg=2.33, fifteen_min_avg=1.21, cpu_share=14.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:55:21.121 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:55:23.157 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:55:28.186 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7766036, partition_space_available_1k=73324448, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1846044, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=227712, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52700, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5889524, free=2641280, buffers=274464, cached=7370360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.21, five_min_avg=2.49, fifteen_min_avg=1.33, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:56:21.087 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:56:23.085 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:56:28.111 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7769324, partition_space_available_1k=73321160, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1872584, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=252056, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52720, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5978144, free=2557908, buffers=275348, cached=7364228, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=2.54, fifteen_min_avg=1.42, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:57:21.145 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:57:23.128 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:57:28.159 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7786844, partition_space_available_1k=73303640, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1896168, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=276336, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52740, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6026448, free=2490664, buffers=276056, cached=7382460, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=2.77, fifteen_min_avg=1.57, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:58:21.095 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:58:23.156 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:58:28.176 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7795516, partition_space_available_1k=73294968, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1912964, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=301012, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52740, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6070504, free=2431644, buffers=276560, cached=7396920, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.12, five_min_avg=2.8, fifteen_min_avg=1.66, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 00:59:21.116 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 00:59:23.132 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 00:59:28.155 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7804644, partition_space_available_1k=73285840, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1946168, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=326156, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52748, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6119536, free=2377456, buffers=276932, cached=7401704, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.04, fifteen_min_avg=1.82, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:00:21.165 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:00:23.236 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:00:28.265 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7819028, partition_space_available_1k=73271456, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1978608, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=247472, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52756, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6055856, free=2430716, buffers=277200, cached=7411856, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.51, five_min_avg=3.12, fifteen_min_avg=1.92, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:01:21.120 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:01:23.034 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:01:28.056 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7827024, partition_space_available_1k=73263460, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2009068, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=252900, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52796, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6102608, free=2370596, buffers=277348, cached=7425076, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.86, five_min_avg=3.28, fifteen_min_avg=2.06, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:02:21.127 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:02:23.181 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:02:28.207 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7835176, partition_space_available_1k=73255308, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2042992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258308, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52808, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6137764, free=2326632, buffers=277480, cached=7433752, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.95, five_min_avg=3.45, fifteen_min_avg=2.19, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:03:21.125 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:03:23.224 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:03:28.254 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7843600, partition_space_available_1k=73246884, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2091628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=261440, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52820, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6204812, free=2244908, buffers=277572, cached=7448336, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.28, five_min_avg=3.68, fifteen_min_avg=2.36, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:04:21.146 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:04:23.120 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:04:28.141 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7853960, partition_space_available_1k=73236524, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2154960, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=266644, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52816, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6252868, free=2180888, buffers=277640, cached=7464232, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=3.74, fifteen_min_avg=2.46, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:05:21.171 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:05:23.184 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:05:28.217 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7868252, partition_space_available_1k=73222232, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2171820, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=273772, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52816, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6275580, free=2153116, buffers=277660, cached=7469272, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.13, five_min_avg=3.54, fifteen_min_avg=2.47, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:06:21.158 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:06:22.961 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:06:27.984 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7882160, partition_space_available_1k=73208324, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2183664, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=276816, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52828, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6280508, free=2133508, buffers=277676, cached=7483936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.48, five_min_avg=3.9, fifteen_min_avg=2.67, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:07:21.155 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:07:23.098 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:07:28.120 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7888656, partition_space_available_1k=73201828, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2190340, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=282016, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52828, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6279384, free=2122868, buffers=277688, cached=7495688, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.24, five_min_avg=4.19, fifteen_min_avg=2.84, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:08:21.153 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:08:23.256 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:08:28.277 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7897852, partition_space_available_1k=73192632, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2196280, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=287236, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52836, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6308200, free=2108292, buffers=277688, cached=7481448, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.63, five_min_avg=3.97, fifteen_min_avg=2.86, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:09:21.170 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:09:23.226 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:09:28.249 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7892788, partition_space_available_1k=73197696, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2176320, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=291004, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52836, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6288776, free=2113708, buffers=277704, cached=7495440, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.7, five_min_avg=3.94, fifteen_min_avg=2.92, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:10:21.186 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:10:23.270 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:10:28.301 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7897352, partition_space_available_1k=73193132, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2178104, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=292628, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52844, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6307320, free=2085428, buffers=277712, cached=7505168, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.77, five_min_avg=3.93, fifteen_min_avg=2.98, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:11:21.192 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:11:23.320 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:11:28.342 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7907140, partition_space_available_1k=73183344, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2183052, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=297904, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52844, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6311236, free=2076292, buffers=277712, cached=7510388, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=3.69, fifteen_min_avg=2.96, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:12:21.220 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:12:23.232 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:12:28.255 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7915984, partition_space_available_1k=73174500, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2187568, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=301564, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52848, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6325820, free=2052560, buffers=277712, cached=7519536, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.0, five_min_avg=3.55, fifteen_min_avg=2.96, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:13:21.189 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:13:23.262 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:13:28.285 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7921092, partition_space_available_1k=73169392, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2199956, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=306820, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52852, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6352056, free=2016268, buffers=277716, cached=7529588, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.4, five_min_avg=3.57, fifteen_min_avg=3.0, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:14:21.188 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:14:23.159 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:14:28.182 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7930684, partition_space_available_1k=73159800, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2204432, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=312252, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52852, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6330824, free=2027492, buffers=277720, cached=7539592, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=3.53, fifteen_min_avg=3.02, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:15:21.166 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:15:23.265 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:15:28.296 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7940124, partition_space_available_1k=73150360, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2204052, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=316436, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52856, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6354384, free=1994184, buffers=277728, cached=7549332, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.62, fifteen_min_avg=3.09, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:16:21.163 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:16:23.236 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:16:28.259 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7954148, partition_space_available_1k=73136336, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2208008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=321568, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52860, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6359940, free=1984256, buffers=277728, cached=7553704, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.86, five_min_avg=3.66, fifteen_min_avg=3.14, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:17:21.171 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:17:23.293 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:17:28.318 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7954412, partition_space_available_1k=73136072, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2206704, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=326940, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52864, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6331464, free=2002472, buffers=277736, cached=7563956, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=3.64, fifteen_min_avg=3.16, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:18:21.228 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:18:23.227 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:18:28.248 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7968400, partition_space_available_1k=73122084, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2207380, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=330808, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52872, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6364104, free=1960248, buffers=277740, cached=7573536, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=3.65, fifteen_min_avg=3.2, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:19:21.173 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:19:23.093 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:19:28.114 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7973120, partition_space_available_1k=73117364, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2205512, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=336480, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52872, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6369552, free=1954784, buffers=277740, cached=7573552, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.27, five_min_avg=3.83, fifteen_min_avg=3.29, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:20:21.254 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:20:23.405 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:20:28.436 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7977456, partition_space_available_1k=73113028, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2207036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=340292, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53304, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6386772, free=1923204, buffers=277740, cached=7587912, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.83, five_min_avg=3.78, fifteen_min_avg=3.31, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:21:21.245 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:21:23.283 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:21:28.317 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7987264, partition_space_available_1k=73103220, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2212060, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=345616, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53308, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6380068, free=1920056, buffers=277740, cached=7597764, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.3, five_min_avg=3.63, fifteen_min_avg=3.28, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:22:21.217 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:22:23.363 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:22:28.385 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7997880, partition_space_available_1k=73092604, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2209060, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=351000, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53312, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6388632, free=1905648, buffers=277744, cached=7603604, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.41, five_min_avg=3.61, fifteen_min_avg=3.3, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:23:21.215 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:23:23.361 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:23:28.382 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8002928, partition_space_available_1k=73087556, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2209580, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355296, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53312, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6383736, free=1900388, buffers=277752, cached=7613752, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.05, five_min_avg=3.47, fifteen_min_avg=3.26, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:24:21.223 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:24:23.032 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:24:28.055 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8011240, partition_space_available_1k=73079244, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2221452, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=360728, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53320, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6377400, free=1902808, buffers=277756, cached=7617664, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.46, fifteen_min_avg=3.27, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:25:21.260 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:25:23.239 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:25:28.267 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8016688, partition_space_available_1k=73073796, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2209640, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=364368, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53324, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6378680, free=1891216, buffers=277756, cached=7627976, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.3, five_min_avg=3.45, fifteen_min_avg=3.28, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:26:21.275 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:26:23.590 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:26:28.618 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8027260, partition_space_available_1k=73063224, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2212124, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=370188, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53328, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6388564, free=1875664, buffers=277756, cached=7633644, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.61, five_min_avg=3.51, fifteen_min_avg=3.31, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:27:21.274 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:27:23.442 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:27:28.476 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8031632, partition_space_available_1k=73058852, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2211268, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=374640, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53328, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6399136, free=1855136, buffers=277756, cached=7643600, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=3.48, fifteen_min_avg=3.32, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:28:21.285 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:28:23.384 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:28:28.407 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8041640, partition_space_available_1k=73048844, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2213140, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=382728, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53336, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6405616, free=1843304, buffers=277756, cached=7648952, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=3.51, fifteen_min_avg=3.34, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:29:21.288 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:29:23.360 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:29:28.382 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8047596, partition_space_available_1k=73042888, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2211752, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=388564, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53336, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6384324, free=1853456, buffers=277760, cached=7660088, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.96, five_min_avg=3.76, fifteen_min_avg=3.44, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:30:21.333 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:30:23.495 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:30:28.538 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8058412, partition_space_available_1k=73032072, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=392924, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53344, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6418292, free=1813604, buffers=277760, cached=7665972, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.32, five_min_avg=3.65, fifteen_min_avg=3.43, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:31:21.335 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:31:23.465 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:31:28.487 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8064300, partition_space_available_1k=73026184, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2213224, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=398228, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53344, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6425044, free=1796004, buffers=277760, cached=7676820, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=3.67, fifteen_min_avg=3.45, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:32:21.304 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:32:23.478 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:32:28.501 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8072872, partition_space_available_1k=73017612, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2213436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=402556, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53348, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6427788, free=1784144, buffers=277760, cached=7685936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.92, five_min_avg=3.77, fifteen_min_avg=3.5, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:33:21.311 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:33:23.524 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:33:28.546 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8083536, partition_space_available_1k=73006948, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214224, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=408428, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53352, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6433668, free=1772264, buffers=277760, cached=7691936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=3.79, fifteen_min_avg=3.52, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:34:21.322 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:34:23.405 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:34:28.431 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8099824, partition_space_available_1k=72990660, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2213804, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=412748, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53356, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6386808, free=1802260, buffers=277764, cached=7708796, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.65, five_min_avg=3.78, fifteen_min_avg=3.54, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:35:21.371 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:35:23.413 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:35:28.444 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8104460, partition_space_available_1k=72986024, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2214088, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=418156, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53360, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6349764, free=1825024, buffers=277764, cached=7723076, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=3.97, fifteen_min_avg=3.62, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:36:21.339 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:36:23.346 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:36:28.368 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8119672, partition_space_available_1k=72970812, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=422324, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53368, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6374392, free=1799532, buffers=277764, cached=7723940, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.29, five_min_avg=3.78, fifteen_min_avg=3.58, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:37:21.365 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:37:23.329 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:37:28.351 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8119904, partition_space_available_1k=72970580, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216940, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=428152, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53368, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6383056, free=1780532, buffers=277764, cached=7734276, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.73, fifteen_min_avg=3.58, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:38:21.354 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:38:23.427 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:38:28.450 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8134292, partition_space_available_1k=72956192, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216980, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=432460, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53372, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6397420, free=1756204, buffers=277764, cached=7744240, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=3.73, fifteen_min_avg=3.59, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:39:21.338 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:39:23.349 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:39:28.374 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8139232, partition_space_available_1k=72951252, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216348, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=437832, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53372, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6393092, free=1750480, buffers=277764, cached=7754292, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=3.63, fifteen_min_avg=3.56, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:40:21.346 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:40:23.249 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:40:28.281 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8149540, partition_space_available_1k=72940944, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2217412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=442308, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53380, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6370876, free=1771432, buffers=277764, cached=7755556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=3.71, fifteen_min_avg=3.59, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:41:21.369 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:41:23.358 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:41:28.381 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8155752, partition_space_available_1k=72934732, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2219932, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=447992, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53380, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6398184, free=1728332, buffers=277764, cached=7771348, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.54, five_min_avg=3.63, fifteen_min_avg=3.57, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:42:21.346 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:42:23.373 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:42:28.394 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8161948, partition_space_available_1k=72928536, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2219912, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=452376, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53380, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6394364, free=1725812, buffers=277764, cached=7777688, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.53, five_min_avg=3.61, fifteen_min_avg=3.56, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:43:21.354 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:43:23.435 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:43:28.466 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8172140, partition_space_available_1k=72918344, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2220584, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=458176, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53388, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6417792, free=1701844, buffers=277764, cached=7778228, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.49, fifteen_min_avg=3.52, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:44:21.365 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:44:23.397 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:44:28.420 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8172616, partition_space_available_1k=72917868, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2219800, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=462480, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53388, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6407212, free=1697140, buffers=277768, cached=7793508, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.43, fifteen_min_avg=3.5, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:45:21.382 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:45:23.491 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:45:28.520 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8187192, partition_space_available_1k=72903292, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2221988, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=467808, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53400, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6422096, free=1677104, buffers=277768, cached=7798660, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.06, five_min_avg=3.34, fifteen_min_avg=3.46, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:46:21.363 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:46:23.526 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:46:28.550 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8194484, partition_space_available_1k=72896000, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2196628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=472152, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53400, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6393660, free=1697940, buffers=277768, cached=7806260, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.67, five_min_avg=3.44, fifteen_min_avg=3.49, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:47:21.381 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:47:23.610 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:47:28.631 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8200092, partition_space_available_1k=72890392, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2207840, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=477936, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53408, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6413508, free=1667360, buffers=277772, cached=7816988, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.93, five_min_avg=3.52, fifteen_min_avg=3.51, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:48:21.381 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:48:23.432 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:48:28.454 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8208176, partition_space_available_1k=72882308, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2208564, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=482168, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53408, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6409284, free=1663280, buffers=277780, cached=7825284, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.44, five_min_avg=3.48, fifteen_min_avg=3.5, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:49:21.377 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:49:23.431 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:49:28.453 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8218608, partition_space_available_1k=72871876, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2208428, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=487476, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53412, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6415332, free=1656332, buffers=277780, cached=7826184, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=3.42, fifteen_min_avg=3.47, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:50:21.370 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:50:23.348 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:50:28.377 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8229228, partition_space_available_1k=72861256, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2208460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=491740, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53416, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6395988, free=1660076, buffers=277792, cached=7841772, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.45, fifteen_min_avg=3.48, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:51:21.360 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:51:23.491 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:51:28.520 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8235192, partition_space_available_1k=72855292, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2210036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=497112, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53420, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6408732, free=1641100, buffers=277792, cached=7848004, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.24, five_min_avg=3.38, fifteen_min_avg=3.45, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:52:21.356 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:52:23.336 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:52:28.358 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8239364, partition_space_available_1k=72851120, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2210544, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=501668, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53424, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6402560, free=1632972, buffers=277792, cached=7862304, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.1, five_min_avg=3.31, fifteen_min_avg=3.42, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:53:21.380 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:53:23.436 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:53:28.458 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8253336, partition_space_available_1k=72837148, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2210900, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=506528, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53428, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6407520, free=1628420, buffers=277792, cached=7861896, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.21, five_min_avg=3.29, fifteen_min_avg=3.41, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:54:21.386 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:54:23.441 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:54:28.462 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8253112, partition_space_available_1k=72837372, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2210888, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=511696, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53428, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6410256, free=1615972, buffers=277796, cached=7871604, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=3.37, fifteen_min_avg=3.43, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:55:21.485 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:55:23.391 7f5f43845df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:55:28.421 7f5f43845df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8266632, partition_space_available_1k=72823852, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2210724, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=516232, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53436, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6410032, free=1607680, buffers=277800, cached=7880116, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.51, five_min_avg=3.4, fifteen_min_avg=3.44, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:56:04.555 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
02/10/2025 01:56:04.555 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
02/10/2025 01:56:04.555 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
02/10/2025 01:56:04.559 7f61a09ce748 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
02/10/2025 01:56:04.560 7f61a09ce748 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
02/10/2025 01:56:04.560 7f619f5e4048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
02/10/2025 01:56:04.561 7f619f5e4048 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 01:56:04.562 7f619f5e4048 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 01:56:04.562 7f619f5e4048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
02/10/2025 01:56:04.562 7f619f5e4048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:56:04.563 7f619f5e4048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 01:56:04.563 7f619f5e4048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:56:04.563 7f619f5e4048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:56:04.563 7f619f5e4048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip =   collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
02/10/2025 01:56:04.563 7f619f5e4048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:56:08.293 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 28811216344393028925944468355843025010761566813416766431978609851851944804363, 'start': '1739152562000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 55869751347553544739022065612541377075995567379996604790143112705088710265685, 'start': '1739152563000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 11736851940611091465110674595036301562930910179945874258079779730074413523049, 'start': '1739152562000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
02/10/2025 01:56:08.578 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 01:56:08.580 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 01:56:08.581 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 01:56:08.581 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:08.581 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:08.943 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:08.944 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739152568944468, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.12:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 01:56:08.944 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739152568944808, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152562000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152563000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152562000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 01:56:08.959 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:08.959 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:10.716 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:56:12.106 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 01:56:12.107 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:14.667 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
02/10/2025 01:56:14.668 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739152574668110, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.12:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 01:56:14.669 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:56:18.304 7f619f5a4248 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
02/10/2025 01:56:18.305 7f619f5a4248 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/10/2025 01:56:18.306 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
02/10/2025 01:56:18.306 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
02/10/2025 01:56:18.306 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 01:56:18.306 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 7> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 01:56:18.306 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 2> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 01:56:18.306 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
02/10/2025 01:56:18.307 7f619f5e4148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/10/2025 01:56:18.308 7f619f5e4148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/10/2025 01:56:19.858 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:56:28.500 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
02/10/2025 01:56:32.617 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
02/10/2025 01:56:33.697 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
02/10/2025 01:56:34.911 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
02/10/2025 01:56:40.316 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
02/10/2025 01:56:45.571 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
02/10/2025 01:56:50.596 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739152610596473, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152562000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152563000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152562000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 01:56:50.597 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8871416, partition_space_available_1k=72219068, percentage_partition_space_used=11, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1297008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=288084, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44640, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5209468, free=2014748, buffers=295328, cached=8656084, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=12.66, five_min_avg=5.92, fifteen_min_avg=4.29, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:57:09.435 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:57:10.533 7f61a65aef20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739152630533749, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.12:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 01:57:12.440 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:57:17.873 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=9420580, partition_space_available_1k=71669904, percentage_partition_space_used=12, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1328700, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=341616, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44652, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5297120, free=1411144, buffers=353144, cached=9114220, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.15, five_min_avg=5.98, fifteen_min_avg=4.37, cpu_share=18.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:58:09.252 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:58:13.012 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:58:18.898 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10488940, partition_space_available_1k=70601544, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1688768, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=420020, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44652, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5766984, free=392028, buffers=380988, cached=9635628, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.92, five_min_avg=5.79, fifteen_min_avg=4.4, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:58:34.873 7f619f5a4248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
02/10/2025 01:58:34.873 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established]
02/10/2025 01:58:34.877 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 01:58:34.878 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect]
02/10/2025 01:58:34.878 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:34.878 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Established to Connect on EvTcpClose  >>,  ]  description = Collector:Collector[Established to Connect on EvTcpClose] connection down  >>,  ]  >>  
02/10/2025 01:58:34.878 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:34.878 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 01:58:34.879 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 01:58:34.879 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 01:58:34.880 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:58:34.880 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:58:34.898 7f619f5a4248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
02/10/2025 01:58:34.898 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit]
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle]
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = ClientInit to Idle on EvTcpClose  >>,  ]  description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down  >>,  ]  >>  
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 2  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:58:34.899 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:38.902 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:58:38.903 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 01:58:38.903 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:58:38.903 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:38.903 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:58:38.903 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:38.903 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 01:58:38.904 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 01:58:38.904 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 01:58:38.904 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:58:38.904 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 01:58:47.442 7f619f5a4248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
02/10/2025 01:58:47.442 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit]
02/10/2025 01:58:47.442 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle]
02/10/2025 01:58:47.443 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:47.443 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = ClientInit to Idle on EvTcpClose  >>,  ]  description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down  >>,  ]  >>  
02/10/2025 01:58:47.443 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:47.443 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:47.443 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:58:47.443 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:51.446 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:58:51.447 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:58:51.447 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:58:51.447 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:51.447 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:58:51.447 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:51.447 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:58:51.448 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:55.451 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:58:55.452 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:58:55.455 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:58:55.455 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:55.455 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:58:55.455 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:55.456 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:58:55.456 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:58:55.456 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:55.456 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:58:55.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:55.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:55.457 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:58:55.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:59.453 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:58:59.457 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:58:59.458 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:59.458 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:58:59.458 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:59.458 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:58:59.458 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:58:59.458 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:03.458 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:03.459 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:03.459 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:03.459 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:03.460 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:03.461 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:07.461 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:07.462 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:07.463 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:07.463 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:07.463 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:07.463 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:07.463 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:07.463 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:07.464 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:07.464 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:07.464 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:07.464 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:07.464 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:07.464 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:09.156 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 01:59:09.367 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:09.367 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 10439116  disk_space_available_1k = 70651368  config_db_size_1k = 3784  >>,  ]  >>  
02/10/2025 01:59:09.367 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:10.696 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 01:59:11.464 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:11.466 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:11.466 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:11.466 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:11.466 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:11.466 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:11.466 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:11.467 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:11.467 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:11.467 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:11.467 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:11.467 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:11.468 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:11.468 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:15.466 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:15.468 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:15.468 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:15.468 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:15.468 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:15.468 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:15.468 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:15.469 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:15.825 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10439132, partition_space_available_1k=70651352, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1703440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=422920, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44704, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=4138264, free=2125548, buffers=387092, cached=9524724, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=5.03, fifteen_min_avg=4.22, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 01:59:15.825 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:15.825 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 10439132  partition_space_available_1k = 70651352  percentage_partition_space_used = 13  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1703440  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 422920  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 44704  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 4138264  free = 2125548  buffers = 387092  cached = 9524724  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 3.52  five_min_avg = 5.03  fifteen_min_avg = 4.22  cpu_share = 4.75  node_type = config-database-node  >>  >>  
02/10/2025 01:59:15.826 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:19.470 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:19.471 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:19.471 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:19.471 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:19.471 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:19.471 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:19.472 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:23.473 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:23.475 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:23.475 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:23.475 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:23.476 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:23.477 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:23.477 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:27.475 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:27.477 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:27.477 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:27.477 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:27.477 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:27.477 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:27.477 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:27.478 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:31.478 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:31.480 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:31.480 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:31.480 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:31.480 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:31.480 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:31.480 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:31.481 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:35.482 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:35.484 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:35.484 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:35.484 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:35.484 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:35.484 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:35.484 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:35.485 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:35.485 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:35.485 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:35.485 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:35.485 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:35.486 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:35.486 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:39.487 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:39.488 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:39.488 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:39.488 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:39.488 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:39.488 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:39.488 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:39.489 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:43.488 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:43.490 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:43.490 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:43.490 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:43.490 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:43.490 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:43.490 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:43.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:47.490 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:47.491 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:47.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:47.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:47.492 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:47.493 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:47.493 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:51.491 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:51.493 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:51.494 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:51.494 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:51.494 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:51.494 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:51.494 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:51.494 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:55.494 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:55.496 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:55.497 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:55.497 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:55.497 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:55.497 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:55.497 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:55.497 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:59.496 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 01:59:59.497 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 01:59:59.497 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 01:59:59.498 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:59.499 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 01:59:59.499 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 01:59:59.499 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:03.500 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 02:00:03.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:03.503 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 02:00:03.503 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:03.503 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:03.503 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:03.503 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:07.502 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:07.503 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 02:00:07.503 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:07.503 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:07.504 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:07.504 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:07.504 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 02:00:07.504 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 02:00:07.505 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:07.505 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 02:00:07.505 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:07.505 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:07.505 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:07.506 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:09.163 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:00:09.388 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:09.388 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 10467656  disk_space_available_1k = 70622828  config_db_size_1k = 3784  >>,  ]  >>  
02/10/2025 02:00:09.389 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:10.738 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:00:11.504 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:11.506 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 02:00:11.506 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:11.506 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:11.506 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:11.507 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:11.507 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 02:00:11.507 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 02:00:11.507 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:11.508 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 02:00:11.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:11.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:11.508 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:11.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:15.507 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:15.508 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 02:00:15.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:15.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:15.508 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:15.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:15.508 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:15.509 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:16.701 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10467456, partition_space_available_1k=70623028, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1700300, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=422964, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44796, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=4872560, free=1314888, buffers=391020, cached=9597160, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=4.66, fifteen_min_avg=4.14, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:00:16.701 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:16.701 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 10467456  partition_space_available_1k = 70623028  percentage_partition_space_used = 13  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1700300  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 422964  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 44796  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175628  used = 4872560  free = 1314888  buffers = 391020  cached = 9597160  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 3.49  five_min_avg = 4.66  fifteen_min_avg = 4.14  cpu_share = 7.25  node_type = config-database-node  >>  >>  
02/10/2025 02:00:16.701 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:19.510 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:19.512 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 02:00:19.512 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:19.512 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:19.512 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:19.512 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:19.512 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.17:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:19.513 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:23.514 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:23.515 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/10/2025 02:00:23.515 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:23.516 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.30:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/10/2025 02:00:23.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:23.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:23.517 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 3  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.30:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:23.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:27.516 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:27.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 02:00:27.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:27.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:27.517 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:27.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:27.517 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 02:00:27.518 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 02:00:27.518 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 02:00:27.518 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 02:00:27.518 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 02:00:29.728 7f619f5a4248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit]
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle]
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Down  description = ClientInit to Idle on EvTcpClose  >>,  ]  description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down  >>,  ]  >>  
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 4  pid = 1  http_port = 8112  start_time = 1739152564562366  collector_name =   collector_ip = 10.0.0.12:8086  collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 5  >>  
02/10/2025 02:00:29.729 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:00:33.732 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:00:33.733 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 02:00:33.735 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:00:33.735 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:33.738 7f619f5e4448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.17:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:00:33.738 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:00:33.738 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 02:00:33.738 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 02:00:33.738 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 02:00:33.739 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 02:00:33.739 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 02:00:33.750 7f619f5a4248 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017
02/10/2025 02:00:33.753 7f619f5a4248 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/10/2025 02:00:33.754 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
02/10/2025 02:00:33.754 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
02/10/2025 02:00:33.754 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 5> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 02:00:33.755 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 74> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:00:33.755 7f619f5e4448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 34> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 02:00:33.756 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
02/10/2025 02:00:33.758 7f619f5e4148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/10/2025 02:00:33.760 7f619f5e4148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/10/2025 02:00:39.764 7f619f5a4248 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438
02/10/2025 02:00:39.764 7f619f5a4248 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/10/2025 02:00:39.765 7f619f5e4148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
02/10/2025 02:00:39.765 7f619f5e4148 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
02/10/2025 02:00:39.765 7f619f5e4148 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 5> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 02:00:39.765 7f619f5e4148 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 75> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:00:39.765 7f619f5e4148 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 35> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 02:00:39.765 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/10/2025 02:00:39.769 7f619f5e4448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/10/2025 02:01:09.187 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:01:10.793 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:01:16.715 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10604168, partition_space_available_1k=70486316, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1700928, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=425600, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44868, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6015500, free=463228, buffers=397292, cached=9299608, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=4.46, fifteen_min_avg=4.1, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:02:09.201 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:02:10.922 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:02:16.810 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11038068, partition_space_available_1k=70052416, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1702092, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=425812, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44880, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6002260, free=428424, buffers=407944, cached=9337000, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.17, five_min_avg=4.62, fifteen_min_avg=4.17, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:03:09.202 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:03:10.766 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:03:15.807 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11043348, partition_space_available_1k=70047136, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1708084, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=425688, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44884, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6061008, free=365428, buffers=407964, cached=9341228, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.48, five_min_avg=3.93, fifteen_min_avg=3.96, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:04:09.220 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:04:10.908 7f61a65aef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:04:16.835 7f61a65aef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11047376, partition_space_available_1k=70043108, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1734640, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=437300, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44888, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6143740, free=377940, buffers=406740, cached=9247208, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.09, five_min_avg=3.51, fifteen_min_avg=3.81, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:04:43.991 7f619f5a4448 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_process_state:zookeeper,group:zookeeper,state:PROCESS_STATE_EXITED
02/10/2025 02:04:43.992 7f619f5a4448 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739153083992182, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152562000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1739152563000000', last_stop_time='', last_exit_time='1739153083992105', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739152562000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-06 01:11:07.385283", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7691-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 02:05:14.044 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
02/10/2025 02:05:14.044 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
02/10/2025 02:05:14.044 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
02/10/2025 02:05:14.058 7f346a2e9a20 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
02/10/2025 02:05:14.058 7f346a2e9a20 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
02/10/2025 02:05:14.059 7f346a2f1390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
02/10/2025 02:05:14.060 7f346a2f1390 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:05:14.061 7f346a2f1390 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 02:05:14.061 7f346a2f1390 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
02/10/2025 02:05:14.061 7f346a2f1390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.061 7f346a2f1390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 02:05:14.061 7f346a2f1390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.061 7f346a2f1390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.062 7f346a2f1390 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1739153114061207  collector_name =   collector_ip =   collector_list = [ 10.0.0.12:8086, 10.0.0.17:8086, 10.0.0.30:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
02/10/2025 02:05:14.062 7f346a2f1390 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.323 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 68622345665491835589603098213250291814304020208282765622017239352148211158051, 'start': '1739153110000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 76336562711303879719662907522560095704105103447913483987936782167961996658315, 'start': '1739153110000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 67539789213650729950763234425652475160817369362523655422644754175126426155428, 'start': '1739153111000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
02/10/2025 02:05:14.573 7f3472bd7a20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.574 7f3472bd7a20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-10 00:48:08.671999", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7708-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = nightly-nightly.el7  installed_package_version = nightly-nightly.el7  >>  
02/10/2025 02:05:14.574 7f3472bd7a20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.574 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739153114574804, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 02:05:14.575 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739153114575106, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739153110000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739153110000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739153111000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-10 00:48:08.671999", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7708-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 02:05:14.576 7f3469a19438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.577 7f3469a19438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 02:05:14.577 7f3469a19438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.577 7f3469a19438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:14.577 7f3469a19438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739153110000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739153110000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1739153111000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-10 00:48:08.671999", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7708-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
02/10/2025 02:05:14.577 7f3469a19438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:15.214 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:05:15.913 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2705-2., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 02:05:15.914 7f346a681588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:15.914 7f346a681588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  database_usage = [ <<  disk_space_used_1k = 11943560  disk_space_available_1k = 69146924  config_db_size_1k = 4732  >>,  ]  >>  
02/10/2025 02:05:15.914 7f346a681588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:17.186 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
02/10/2025 02:05:17.186 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739153117186768, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 02:05:17.188 7f346c313048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:17.188 7f346c313048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/10/2025 02:05:17.188 7f346c313048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/10/2025 02:05:18.060 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/10/2025 02:05:18.061 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2705-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.12:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/10/2025 02:05:18.062 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/10/2025 02:05:18.063 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 02:05:18.063 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/10/2025 02:05:18.071 7f346a2ed198 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017
02/10/2025 02:05:18.071 7f346a2ed198 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/10/2025 02:05:18.072 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
02/10/2025 02:05:18.072 7f3469a45c18 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
02/10/2025 02:05:18.072 7f3469a45c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/10/2025 02:05:18.072 7f3469a45c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 7> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:05:18.072 7f3469a45c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 2> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/10/2025 02:05:18.072 7f3469a45c18 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
02/10/2025 02:05:18.073 7f3469a32390 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/10/2025 02:05:18.073 7f3469a32390 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/10/2025 02:05:20.345 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:05:22.537 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
02/10/2025 02:05:24.236 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
02/10/2025 02:05:25.626 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
02/10/2025 02:05:26.537 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
02/10/2025 02:05:27.814 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
02/10/2025 02:05:28.748 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
02/10/2025 02:05:33.778 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739153133778643, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739153110000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739153110000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1739153111000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-10 00:48:08.671999", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7708-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 02:05:33.779 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12247356, partition_space_available_1k=68843128, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1248156, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=340764, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5775672, free=340852, buffers=430176, cached=9628928, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=3.42, fifteen_min_avg=3.74, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:06:14.895 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:06:16.007 7f346c97cb08 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2705-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1739153176007365, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.12:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/10/2025 02:06:17.158 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:06:22.182 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12252356, partition_space_available_1k=68838128, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1310140, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=348400, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52488, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=5767864, free=365780, buffers=432752, cached=9609232, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.98, five_min_avg=3.33, fifteen_min_avg=3.7, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:07:14.840 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:07:16.971 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:07:22.015 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12257148, partition_space_available_1k=68833336, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1689412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=352080, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53456, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6307488, free=352156, buffers=430660, cached=9085324, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.82, five_min_avg=2.92, fifteen_min_avg=3.53, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:08:14.834 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:08:16.527 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:08:21.866 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12251636, partition_space_available_1k=68838848, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1673260, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=352164, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53392, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6477664, free=357932, buffers=430508, cached=8909524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.02, five_min_avg=2.49, fifteen_min_avg=3.34, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:09:14.869 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:09:16.530 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:09:21.880 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12254728, partition_space_available_1k=68835756, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1675296, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=352248, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53388, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6518380, free=373932, buffers=430352, cached=8852964, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.86, five_min_avg=2.18, fifteen_min_avg=3.18, cpu_share=5.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:10:14.834 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:10:16.514 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:10:21.891 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12258316, partition_space_available_1k=68832168, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1695064, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=352284, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53408, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6540904, free=358672, buffers=429820, cached=8846232, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.93, five_min_avg=1.95, fifteen_min_avg=3.03, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:11:14.852 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:11:16.475 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:11:21.906 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12264936, partition_space_available_1k=68825548, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1682792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=352304, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53404, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6546792, free=368164, buffers=429056, cached=8831616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.01, five_min_avg=1.79, fifteen_min_avg=2.91, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:12:14.860 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:12:16.513 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:12:21.909 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12268236, partition_space_available_1k=68822248, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1685144, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=352540, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53396, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6569008, free=369808, buffers=428904, cached=8807908, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.06, five_min_avg=1.66, fifteen_min_avg=2.79, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:13:14.920 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:13:16.532 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:13:21.914 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12271428, partition_space_available_1k=68819056, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1684300, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355496, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53384, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6605680, free=363160, buffers=428488, cached=8778300, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.84, five_min_avg=1.48, fifteen_min_avg=2.66, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:14:14.884 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:14:16.503 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:14:21.915 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12274568, partition_space_available_1k=68815916, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1686216, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355500, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53388, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6595508, free=370156, buffers=428488, cached=8781476, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.77, five_min_avg=1.34, fifteen_min_avg=2.54, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:15:14.890 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:15:16.526 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:15:21.934 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12277328, partition_space_available_1k=68813156, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1690616, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355524, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53380, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6592764, free=392020, buffers=428408, cached=8762436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.83, five_min_avg=1.26, fifteen_min_avg=2.43, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:16:14.901 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:16:16.481 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:16:21.941 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12281440, partition_space_available_1k=68809044, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1699044, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355528, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53372, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6626136, free=376140, buffers=428384, cached=8744968, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.82, five_min_avg=1.17, fifteen_min_avg=2.32, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:17:14.910 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:17:16.677 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:17:21.951 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12280080, partition_space_available_1k=68810404, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1691016, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355528, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53376, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6629496, free=380996, buffers=428704, cached=8736432, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.07, five_min_avg=1.16, fifteen_min_avg=2.24, cpu_share=6.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:18:14.902 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:18:16.514 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:18:21.953 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12283788, partition_space_available_1k=68806696, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1691972, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355520, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53380, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6631336, free=376352, buffers=428744, cached=8739196, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.52, five_min_avg=1.22, fifteen_min_avg=2.2, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:18:51.282 7f346de5ae10 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 24> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:19:09.774 7f3468351198 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 24> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:19:14.906 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:19:16.497 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:19:21.955 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12641192, partition_space_available_1k=68449292, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1691976, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355596, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53276, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6660632, free=377628, buffers=433348, cached=8704020, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.29, five_min_avg=1.2, fifteen_min_avg=2.13, cpu_share=5.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:19:24.236 7f3471b70780 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 25> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:19:38.944 7f3468e5eb70 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 25> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:19:53.508 7f34683724e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 25> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:20:07.990 7f346a3c5048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 25> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:20:14.920 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:20:16.574 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:20:21.959 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12644652, partition_space_available_1k=68445832, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1704948, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355592, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53284, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6684552, free=345716, buffers=433388, cached=8711972, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.4, five_min_avg=1.24, fifteen_min_avg=2.08, cpu_share=5.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:20:22.395 7f346c4bf2e8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 26> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:20:37.085 7f346a3c3b70 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 26> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:20:51.877 7f3469a37048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 26> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:21:06.389 7f3469a37048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 26> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:21:14.950 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:21:16.587 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:21:20.777 7f346836dd68 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 26> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:21:21.959 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12648932, partition_space_available_1k=68441552, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1696328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355592, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53288, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6671372, free=362784, buffers=433384, cached=8708088, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.92, five_min_avg=1.13, fifteen_min_avg=1.99, cpu_share=6.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:21:35.078 7f346c44a630 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 27> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:21:49.883 7f346a2f1438 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 27> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:22:04.471 7f346a2ed780 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 27> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:22:14.935 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:22:16.600 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:22:18.854 7f3469a17588 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 27> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:22:21.972 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12652400, partition_space_available_1k=68438084, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1697176, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355588, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53292, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6677432, free=359048, buffers=433368, cached=8705780, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.39, five_min_avg=1.23, fifteen_min_avg=1.97, cpu_share=5.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:22:33.096 7f3468364eb8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 28> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:22:47.582 7f346c4c5828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 28> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:23:02.352 7f346c2a6240 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 28> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:23:14.932 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:23:16.841 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:23:17.062 7f346c2a62e8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 28> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:23:21.966 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12659764, partition_space_available_1k=68430720, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1699736, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355592, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53424, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6673520, free=411608, buffers=433160, cached=8657340, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.06, five_min_avg=1.16, fifteen_min_avg=1.9, cpu_share=5.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:23:31.526 7f3471b70780 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 29> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:23:45.897 7f346c44a630 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 29> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:24:00.592 7f3469a190f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 29> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:24:14.958 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:24:15.429 7f3469a45ac8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 29> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:24:16.612 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:24:21.973 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12675088, partition_space_available_1k=68415396, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1700524, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355596, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53428, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6676932, free=399804, buffers=433308, cached=8665584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.97, five_min_avg=1.11, fifteen_min_avg=1.83, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:24:30.117 7f346c787ac8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 30> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:24:44.427 7f347357c0f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 30> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:24:58.955 7f3469a19ac8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 30> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:25:13.369 7f3468359e10 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 30> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:25:14.938 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:25:16.562 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:25:21.978 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12678396, partition_space_available_1k=68412088, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1698880, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355628, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53436, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6693452, free=386512, buffers=433304, cached=8662360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.46, five_min_avg=1.19, fifteen_min_avg=1.81, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:25:27.997 7f346c4c5828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 31> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:25:42.834 7f3469a48a20 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 31> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:25:57.202 7f3468e5e048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 31> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:26:11.700 7f346a2fee10 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 31> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:26:14.944 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:26:16.570 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:26:21.979 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12681908, partition_space_available_1k=68408576, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1702676, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355648, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53444, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6669912, free=407340, buffers=433300, cached=8665076, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.89, five_min_avg=1.09, fifteen_min_avg=1.74, cpu_share=5.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:26:26.090 7f346de5a390 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 32> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:26:40.934 7f346a301e10 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 32> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:26:55.647 7f3469a45240 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 32> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:27:10.168 7f3469a37978 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 32> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:27:14.950 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:27:16.582 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:27:21.991 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12680264, partition_space_available_1k=68410220, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1701408, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355704, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53444, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6710536, free=367308, buffers=433252, cached=8664532, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.93, five_min_avg=1.06, fifteen_min_avg=1.69, cpu_share=5.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:27:24.603 7f3469a48c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 33> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:27:39.024 7f3469a48c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 33> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:27:53.832 7f346a681588 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 33> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:28:08.663 7f346a390b70 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 33> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:28:14.966 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:28:16.613 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:28:21.988 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12683948, partition_space_available_1k=68406536, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1702344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355700, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53448, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6734744, free=345836, buffers=433112, cached=8661936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.08, five_min_avg=1.08, fifteen_min_avg=1.65, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:28:23.010 7f346a3c3c18 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 34> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:28:37.518 7f347261b240 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 34> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:28:51.875 7f346a3436d8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 34> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:29:06.681 7f346a2ed048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 34> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:29:14.961 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:29:16.598 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:29:21.461 7f346a301ac8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 34> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:29:21.988 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12687356, partition_space_available_1k=68403128, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1705084, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355704, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53452, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6716020, free=366080, buffers=433108, cached=8660420, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.9, five_min_avg=1.03, fifteen_min_avg=1.6, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:29:35.881 7f346c44a630 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 35> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:29:50.421 7f346c2fa048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 35> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:30:04.858 7f346a3c3b70 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 35> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:30:14.988 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:30:16.619 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:30:19.617 7f3468e5e828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 35> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:30:21.992 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12690264, partition_space_available_1k=68400220, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1703512, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355704, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53460, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6730316, free=361804, buffers=433008, cached=8650500, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.03, five_min_avg=1.04, fifteen_min_avg=1.56, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:30:34.202 7f3469a19198 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 36> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:30:48.788 7f3469a17cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 36> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:31:03.257 7f346c4c5828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 36> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:31:14.972 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:31:16.732 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:31:17.760 7f346a19a828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 36> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:31:21.994 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12695224, partition_space_available_1k=68395260, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1707492, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355700, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53464, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6732424, free=369348, buffers=432840, cached=8641016, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.01, five_min_avg=1.03, fifteen_min_avg=1.53, cpu_share=5.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:31:32.484 7f346a2ed4e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 37> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:31:47.565 7f346a19a828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 37> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:32:02.031 7f346a343f60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 37> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:32:15.003 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:32:16.591 7f3468500198 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 37> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:32:16.916 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:32:22.008 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12697848, partition_space_available_1k=68392636, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1708248, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355712, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53528, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6734812, free=391584, buffers=432868, cached=8616364, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.82, five_min_avg=0.97, fifteen_min_avg=1.47, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:32:30.970 7f346a3c5048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 38> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:32:45.740 7f3468351828 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 38> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:33:00.629 7f346c64c048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 38> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:33:15.000 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/10/2025 02:33:15.172 7f346c313e10 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 38> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/10/2025 02:33:16.669 7f346c97cb08 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/10/2025 02:33:22.004 7f346c97cb08 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2705-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12701404, partition_space_available_1k=68389080, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1708352, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=355720, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53536, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175628, used=6745096, free=378364, buffers=432924, cached=8619244, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.13, five_min_avg=1.01, fifteen_min_avg=1.45, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/10/2025 02:33:59.063 7f346c4a62e8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2705-2.: 39> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map