02/20/2025 00:45:23.361 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
02/20/2025 00:45:23.361 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
02/20/2025 00:45:23.361 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
02/20/2025 00:45:23.368 7fdbb018f648 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
02/20/2025 00:45:23.368 7fdbb018f648 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
02/20/2025 00:45:23.369 7fdbb018fe48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
02/20/2025 00:45:23.369 7fdbb018fe48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2748-2., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:45:23.369 7fdbb018fe48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip =   collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
02/20/2025 00:45:23.370 7fdbb018fe48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.415 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 32525691521667142027337586278231366852156215072107249699134871929129093745139, 'start': '1740012321000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 109444591970149464864316476398581120284396971716946636174910126444530346880042, 'start': '1740012321000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 95100985632696908018956133831851374831393813032870702619365803427446113106820, 'start': '1740012321000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
02/20/2025 00:45:23.539 7fdbb02a2148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.540 7fdbb02a2148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-17 01:15:29.742621", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7786-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7  installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7  >>  
02/20/2025 00:45:23.540 7fdbb02a2148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.540 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2748-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1740012323540617, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/20/2025 00:45:23.540 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2748-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1740012323540914, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1740012321000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1740012321000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1740012321000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-17 01:15:29.742621", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7786-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/20/2025 00:45:23.542 7fdbb02a2148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.542 7fdbb02a2148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/20/2025 00:45:23.543 7fdbb02a2148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.543 7fdbb02a2148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.543 7fdbb02a2148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1740012321000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1740012321000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1740012321000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-17 01:15:29.742621", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7786-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
02/20/2025 00:45:23.543 7fdbb02a2148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.750 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:45:23.919 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2748-2., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/20/2025 00:45:23.921 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:23.921 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  database_usage = [ <<  disk_space_used_1k = 4675772  disk_space_available_1k = 76414712  config_db_size_1k = 4  >>,  ]  >>  
02/20/2025 00:45:23.921 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:24.284 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
02/20/2025 00:45:24.284 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2748-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1740012324284520, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/20/2025 00:45:24.285 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:24.286 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
02/20/2025 00:45:24.286 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:25.781 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:45:27.370 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:27.371 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:27.371 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:27.372 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:27.372 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:27.373 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:27.373 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:27.374 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:27.374 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:27.375 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:27.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:27.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:27.375 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:27.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:27.844 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
02/20/2025 00:45:28.625 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
02/20/2025 00:45:29.658 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
02/20/2025 00:45:30.598 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
02/20/2025 00:45:31.372 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:31.373 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:31.373 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:31.374 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:31.374 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:31.374 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:31.374 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:31.374 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:31.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:31.375 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:31.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:31.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:31.375 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:31.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:31.604 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
02/20/2025 00:45:32.430 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
02/20/2025 00:45:35.375 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:35.377 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:35.378 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:35.378 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:35.378 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:35.378 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:35.378 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:35.379 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:37.455 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2748-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1740012337455658, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1740012321000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1740012321000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1740012321000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-17 01:15:29.742621", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7786-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/20/2025 00:45:37.456 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=4927364, partition_space_available_1k=76163120, percentage_partition_space_used=6, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1217224, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55408, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51004, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=1730464, free=9956180, buffers=100104, cached=4388888, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.45, five_min_avg=0.46, fifteen_min_avg=0.16, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:45:37.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:37.456 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1740012321000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1740012321000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1740012321000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-17 01:15:29.742621", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7786-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
02/20/2025 00:45:37.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:37.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:37.456 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 4927364  partition_space_available_1k = 76163120  percentage_partition_space_used = 6  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1217224  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 55408  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51004  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175636  used = 1730464  free = 9956180  buffers = 100104  cached = 4388888  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.45  five_min_avg = 0.46  fifteen_min_avg = 0.16  cpu_share = 0  node_type = config-database-node  >>  >>  
02/20/2025 00:45:37.457 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:39.378 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:39.380 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:39.380 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:39.380 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:39.381 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:39.382 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:39.382 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:43.382 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:43.383 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:43.383 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:43.383 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:43.384 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:43.385 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:47.386 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:47.387 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:47.387 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:47.388 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:47.389 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:47.389 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:51.391 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:51.392 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:51.392 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:51.392 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:51.393 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:51.394 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:55.392 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:55.394 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:55.394 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:55.394 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:55.394 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:55.394 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:55.394 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:55.395 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:59.394 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:45:59.396 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:45:59.396 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:45:59.397 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:59.397 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:45:59.397 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:45:59.398 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:45:59.398 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:45:59.399 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:59.399 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:45:59.399 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:59.399 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:45:59.399 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:45:59.399 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:03.398 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:03.399 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:03.400 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:03.400 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:03.400 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:03.400 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:03.400 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:03.400 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:03.401 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:03.402 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:03.402 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:03.402 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:03.402 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:03.402 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:07.402 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:07.404 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:07.404 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:07.404 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:07.404 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:07.404 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:07.404 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:07.405 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:07.405 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:07.405 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:07.405 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:07.405 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:07.406 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:07.406 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:11.406 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:11.407 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:11.408 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:11.409 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:11.409 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:11.409 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:11.409 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:11.409 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:15.410 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:15.411 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:15.411 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:15.411 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:15.412 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:15.413 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:15.413 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:19.415 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:19.416 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:19.417 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:19.417 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:19.417 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:19.417 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:19.417 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:19.417 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:19.418 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:19.418 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:19.418 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:19.418 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:19.418 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:19.418 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:23.419 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:23.422 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:23.422 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:23.422 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:23.422 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:23.423 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:23.700 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:46:23.847 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:23.847 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  database_usage = [ <<  disk_space_used_1k = 5950136  disk_space_available_1k = 75140348  config_db_size_1k = 1652  >>,  ]  >>  
02/20/2025 00:46:23.847 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:24.310 7fdbb5ae7df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2748-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1740012384310050, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.42:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
02/20/2025 00:46:24.311 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:24.311 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:24.312 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:25.278 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:46:27.422 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:27.424 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:27.424 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:27.424 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:27.424 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:27.424 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:27.425 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:30.302 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=5950548, partition_space_available_1k=75139936, percentage_partition_space_used=7, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327388, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55136, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51368, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=1813156, free=8821956, buffers=174460, cached=5366064, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.74, five_min_avg=0.7, fifteen_min_avg=0.26, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:46:30.302 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:30.303 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 5950548  partition_space_available_1k = 75139936  percentage_partition_space_used = 7  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1327388  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 55136  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51368  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175636  used = 1813156  free = 8821956  buffers = 174460  cached = 5366064  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.74  five_min_avg = 0.7  fifteen_min_avg = 0.26  cpu_share = 7.25  node_type = config-database-node  >>  >>  
02/20/2025 00:46:30.303 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:31.424 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:31.426 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:31.426 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:31.426 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:31.426 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:31.426 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:31.426 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:31.427 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:35.430 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:35.432 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:35.432 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:35.432 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:35.432 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:35.432 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:35.432 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:35.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:35.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:35.433 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:35.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:35.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:35.433 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:35.434 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:39.432 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:39.433 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:39.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:39.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:39.433 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:39.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:39.433 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:39.434 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:43.437 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:43.438 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:43.438 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:43.438 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:43.438 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:43.438 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:43.439 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:47.442 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:47.443 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:47.444 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:47.445 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:47.445 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:47.445 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:51.447 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:51.449 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:51.449 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:51.449 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:51.449 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:51.449 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:51.449 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:51.450 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:55.450 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:55.452 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:55.452 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:55.452 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:55.452 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:55.452 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:55.453 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:55.453 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:55.453 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:55.454 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:55.455 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:55.455 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:55.455 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:55.455 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:59.454 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:46:59.455 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:46:59.455 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:46:59.455 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:46:59.456 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:46:59.457 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:03.459 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:03.460 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:03.460 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:03.460 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:03.461 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:03.462 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:03.462 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:03.462 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:07.462 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:07.463 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:07.464 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:07.464 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:07.464 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:07.464 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:07.464 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:07.464 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:07.465 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:07.465 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:07.465 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:07.465 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:07.465 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:07.465 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:11.466 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:11.468 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:11.468 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:11.468 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:11.468 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:11.468 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:11.468 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:11.469 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:11.469 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:11.470 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:11.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:11.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:11.470 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:11.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:15.469 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:15.470 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:15.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:15.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:15.470 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:15.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:15.470 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:15.471 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:19.471 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:19.472 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:19.473 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:19.473 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:19.473 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:19.473 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:19.473 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:19.473 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:19.474 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:19.474 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:19.474 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:19.474 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:19.474 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:19.474 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:23.474 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:23.475 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:23.475 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:23.476 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:23.476 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:23.476 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:23.476 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:23.477 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:23.477 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:23.477 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:23.477 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:23.477 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:23.477 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:23.478 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:23.758 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:47:23.944 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:23.944 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  database_usage = [ <<  disk_space_used_1k = 6249384  disk_space_available_1k = 74841100  config_db_size_1k = 1764  >>,  ]  >>  
02/20/2025 00:47:23.944 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:26.055 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:47:27.476 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:27.478 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:27.478 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:27.479 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:27.482 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:27.482 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:27.482 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:27.483 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:27.483 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:27.484 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:27.484 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:27.484 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:27.485 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:27.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:31.078 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6249860, partition_space_available_1k=74840624, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1648268, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=65560, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51672, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=2652220, free=7603868, buffers=206464, cached=5713084, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.75, five_min_avg=1.09, fifteen_min_avg=0.42, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:47:31.078 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:31.079 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6249860  partition_space_available_1k = 74840624  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1648268  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 65560  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51672  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175636  used = 2652220  free = 7603868  buffers = 206464  cached = 5713084  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.75  five_min_avg = 1.09  fifteen_min_avg = 0.42  cpu_share = 6.75  node_type = config-database-node  >>  >>  
02/20/2025 00:47:31.079 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:31.479 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:31.480 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:31.480 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:31.480 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:31.481 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:31.482 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:35.482 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:35.484 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:35.484 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:35.484 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:35.484 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:35.484 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:35.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:35.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:35.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:35.485 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:35.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:35.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:35.486 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:35.486 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:39.485 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:39.486 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:39.486 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:39.486 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:39.486 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:39.487 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:39.487 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:39.487 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:39.487 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:39.487 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:39.487 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:39.488 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:39.488 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:39.488 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:43.486 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:43.488 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:43.488 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:43.488 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:43.488 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:43.488 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:43.489 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:47.490 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:47.491 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:47.491 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:47.491 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:47.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:47.493 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:47.493 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:51.492 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:51.493 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:51.493 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:51.494 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:51.495 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:51.495 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:51.495 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:51.495 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:55.494 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:55.496 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:55.496 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:55.496 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:55.496 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:55.496 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:55.496 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:55.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:59.499 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:47:59.501 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:47:59.501 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:47:59.501 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:59.501 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:47:59.501 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:47:59.501 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:47:59.502 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:03.502 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:03.503 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:03.504 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:03.504 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:03.504 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:03.504 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:03.504 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:03.504 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:03.505 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:03.505 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:03.505 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:03.505 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:03.505 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:03.505 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:07.507 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:07.509 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:07.509 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:07.509 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:07.510 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:07.510 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:07.510 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:07.510 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:07.510 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:07.510 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:07.511 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:07.511 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:07.511 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:07.511 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:11.510 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:11.511 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:11.512 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:11.513 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:11.513 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:15.515 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:15.516 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:15.516 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:15.516 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:15.517 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:15.518 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:15.518 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:15.518 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:19.517 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:19.519 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:19.520 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:19.520 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:19.520 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:19.520 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:19.520 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:19.520 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:23.522 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:23.524 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:23.524 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:23.524 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:23.524 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:23.525 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:23.526 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:23.526 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:23.711 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:48:23.886 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:23.886 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  database_usage = [ <<  disk_space_used_1k = 6368680  disk_space_available_1k = 74721804  config_db_size_1k = 1804  >>,  ]  >>  
02/20/2025 00:48:23.886 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:25.172 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:48:27.526 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:27.528 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:27.528 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:27.528 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:27.528 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:27.528 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:27.528 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:27.529 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:27.529 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:27.529 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:27.529 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:27.529 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:27.530 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:27.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:30.195 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6368700, partition_space_available_1k=74721784, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1712860, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84408, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51780, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=2927516, free=7195568, buffers=210144, cached=5842408, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.19, five_min_avg=0.93, fifteen_min_avg=0.41, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:48:30.195 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:30.195 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6368700  partition_space_available_1k = 74721784  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1712860  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 84408  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51780  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175636  used = 2927516  free = 7195568  buffers = 210144  cached = 5842408  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.19  five_min_avg = 0.93  fifteen_min_avg = 0.41  cpu_share = 4.25  node_type = config-database-node  >>  >>  
02/20/2025 00:48:30.195 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:31.528 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:31.529 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:31.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:31.531 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:31.531 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:31.531 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:31.531 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:31.531 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:35.530 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:35.532 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:35.532 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:35.532 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:35.532 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:35.532 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:35.532 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:35.533 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:39.533 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:39.534 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:39.534 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:39.535 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:39.535 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:39.535 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:39.535 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:39.536 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:39.536 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:39.536 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:39.536 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:39.536 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:39.537 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:39.537 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:43.538 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:43.539 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:43.540 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:43.541 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:43.541 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:43.541 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:47.542 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:47.543 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:47.544 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:47.545 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:47.545 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:47.545 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:51.547 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:51.548 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:51.548 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:51.548 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:51.549 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:51.550 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:55.549 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:55.550 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:55.551 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:55.552 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:55.552 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:55.552 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:55.552 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:59.554 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:48:59.556 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:48:59.556 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:48:59.556 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:59.556 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:59.557 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:48:59.558 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:48:59.558 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:03.559 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:03.560 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:03.560 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:03.560 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:03.561 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:03.562 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:03.562 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:07.562 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:07.564 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:07.564 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:07.564 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:07.564 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:07.564 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:07.565 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:11.566 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:11.568 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:11.569 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:11.569 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:11.569 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:11.569 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:11.569 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:15.570 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:15.571 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:15.571 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:15.571 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:15.571 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:15.571 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:15.571 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:15.572 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:19.572 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:19.573 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:19.574 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:19.574 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:19.574 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:19.574 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:19.574 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:19.575 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:23.578 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:23.584 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:23.584 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:23.585 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:23.585 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:23.585 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:23.585 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:23.585 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:23.585 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:23.586 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:23.586 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:23.586 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:23.586 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:23.586 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:23.781 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:49:23.993 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:23.993 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  database_usage = [ <<  disk_space_used_1k = 6757988  disk_space_available_1k = 74332496  config_db_size_1k = 1804  >>,  ]  >>  
02/20/2025 00:49:23.993 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:25.240 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:49:27.586 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:27.587 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:27.587 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:27.588 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:27.589 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:27.589 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:27.589 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:27.589 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:30.261 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6757996, partition_space_available_1k=74332488, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1711464, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84616, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51984, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=3081160, free=6581404, buffers=236176, cached=6276896, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.05, five_min_avg=0.93, fifteen_min_avg=0.44, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:49:30.261 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:30.261 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6757996  partition_space_available_1k = 74332488  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1711464  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 84616  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51984  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175636  used = 3081160  free = 6581404  buffers = 236176  cached = 6276896  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.05  five_min_avg = 0.93  fifteen_min_avg = 0.44  cpu_share = 3.0  node_type = config-database-node  >>  >>  
02/20/2025 00:49:30.261 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:31.588 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:31.590 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:31.590 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:31.591 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:31.592 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:31.592 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:31.592 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:31.592 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:35.594 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:35.596 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:35.596 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:35.597 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:35.597 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:35.597 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:35.597 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:35.597 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:35.597 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:35.598 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:35.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:35.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:35.598 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:35.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:39.597 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:39.598 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:39.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:39.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:39.598 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:39.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:39.598 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:39.599 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:43.601 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:43.603 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:43.603 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:43.604 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:43.605 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:43.605 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:47.607 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:47.609 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:47.610 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:47.610 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:47.610 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:47.610 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:47.610 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:47.610 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:51.610 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:51.611 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:51.611 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:51.612 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:51.613 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:51.613 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:55.611 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:55.613 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:55.613 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:55.613 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:55.613 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:55.613 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:55.613 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:55.614 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:59.616 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:49:59.618 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:49:59.618 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:49:59.618 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:59.618 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:49:59.618 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:49:59.618 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:49:59.619 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:03.622 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:03.623 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:03.623 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:03.623 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:03.623 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:03.623 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:03.623 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:03.624 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:07.626 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:07.627 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:07.627 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:07.628 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:07.629 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:07.629 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:07.629 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:07.629 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:11.628 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:11.629 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:11.629 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:11.630 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:11.631 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:15.631 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:15.632 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:15.632 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:15.632 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:15.633 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:15.633 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:15.633 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:15.633 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:15.633 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:15.633 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:15.634 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:15.634 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:15.634 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:15.634 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:19.634 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:19.636 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:19.636 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:19.636 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:19.636 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:19.636 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:19.636 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:19.637 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:23.637 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:23.639 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:23.640 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:23.640 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:23.640 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:23.640 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:23.640 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:23.641 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:23.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:23.642 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:23.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:23.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:23.643 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:23.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:23.785 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:50:24.024 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:24.024 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  database_usage = [ <<  disk_space_used_1k = 6770564  disk_space_available_1k = 74319920  config_db_size_1k = 1804  >>,  ]  >>  
02/20/2025 00:50:24.024 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:25.294 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:50:27.640 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:27.642 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:27.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:27.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:27.642 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:27.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:27.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:27.643 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:30.321 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6770568, partition_space_available_1k=74319916, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1713472, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84720, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52192, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4424796, free=5214108, buffers=240076, cached=6296656, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.75, five_min_avg=0.85, fifteen_min_avg=0.44, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:50:30.322 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:30.322 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6770568  partition_space_available_1k = 74319916  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1713472  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 84720  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 52192  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175636  used = 4424796  free = 5214108  buffers = 240076  cached = 6296656  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 0.75  five_min_avg = 0.85  fifteen_min_avg = 0.44  cpu_share = 4.0  node_type = config-database-node  >>  >>  
02/20/2025 00:50:30.322 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:31.642 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:31.644 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:31.644 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:31.644 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:31.645 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:31.645 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:31.645 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:31.645 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:31.646 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:31.646 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:31.646 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:31.646 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:31.646 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:31.646 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:35.648 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:35.650 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:35.650 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:35.650 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:35.650 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:35.650 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:35.650 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:35.651 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:39.652 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:39.653 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:39.653 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:39.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:39.655 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:39.655 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:39.655 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:43.654 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:43.656 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:43.656 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:43.656 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:43.656 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:43.656 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:43.656 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:43.657 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:47.658 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:47.659 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:47.659 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:47.660 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:47.661 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:47.661 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:47.661 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:51.663 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:51.664 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:51.664 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:51.664 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:51.665 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:51.666 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:51.666 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:51.666 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:55.665 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:55.667 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:55.667 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:55.667 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:55.667 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:55.667 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:55.667 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.47:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:55.668 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:59.668 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:50:59.670 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:50:59.670 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:50:59.670 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.42:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:59.671 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:50:59.672 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.42:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:50:59.672 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:51:03.674 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:51:03.675 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
02/20/2025 00:51:03.676 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:51:03.676 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:51:03.676 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:51:03.676 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:51:03.676 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
02/20/2025 00:51:03.676 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
02/20/2025 00:51:03.677 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:51:03.677 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.29:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
02/20/2025 00:51:03.677 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:51:03.677 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:51:03.677 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1740012323369814  collector_name =   collector_ip = 10.0.0.29:8086  collector_list = [ 10.0.0.29:8086, 10.0.0.47:8086, 10.0.0.42:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
02/20/2025 00:51:03.677 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
02/20/2025 00:51:07.678 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
02/20/2025 00:51:07.680 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
02/20/2025 00:51:07.680 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
02/20/2025 00:51:07.680 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:51:07.680 7fdbb0155348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2748-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.47:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
02/20/2025 00:51:07.680 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
02/20/2025 00:51:07.681 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
02/20/2025 00:51:07.681 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
02/20/2025 00:51:07.681 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
02/20/2025 00:51:07.682 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/20/2025 00:51:07.682 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
02/20/2025 00:51:18.881 7fdbb02a2148 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
02/20/2025 00:51:18.882 7fdbb02a2148 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/20/2025 00:51:18.883 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
02/20/2025 00:51:18.883 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
02/20/2025 00:51:18.883 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 6> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/20/2025 00:51:18.883 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 185> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:51:18.883 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 87> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/20/2025 00:51:18.884 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
02/20/2025 00:51:18.884 7fdbb0155048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/20/2025 00:51:18.885 7fdbb0155048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/20/2025 00:51:23.760 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:51:25.450 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:51:30.475 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7083388, partition_space_available_1k=74007096, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1717732, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=84992, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52432, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4892412, free=4318676, buffers=253272, cached=6711276, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=1.08, fifteen_min_avg=0.54, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:51:32.495 7fdbb02a2148 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439
02/20/2025 00:51:32.495 7fdbb02a2148 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
02/20/2025 00:51:32.496 7fdbb0155048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
02/20/2025 00:51:32.496 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
02/20/2025 00:51:32.496 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 7> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
02/20/2025 00:51:32.496 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 187> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:51:32.496 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 88> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
02/20/2025 00:51:32.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
02/20/2025 00:51:32.497 7fdbb0155348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
02/20/2025 00:52:23.817 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:52:25.928 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:52:30.952 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7227704, partition_space_available_1k=73862780, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1718748, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=182180, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52596, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5647112, free=3459744, buffers=261904, cached=6806876, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.94, five_min_avg=1.56, fifteen_min_avg=0.75, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:53:23.905 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:53:25.916 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:53:30.940 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7279344, partition_space_available_1k=73811140, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1720356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168628, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52640, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5652080, free=3399812, buffers=263220, cached=6860524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.14, five_min_avg=1.89, fifteen_min_avg=0.92, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:54:14.889 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 189> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:54:23.839 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:54:25.782 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:54:30.809 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7748772, partition_space_available_1k=73341712, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1782224, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=158464, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52720, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5727476, free=2826972, buffers=270608, cached=7350580, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=2.14, fifteen_min_avg=1.06, cpu_share=12.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:54:49.204 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 190> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:55:09.242 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 190> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:55:23.807 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:55:25.924 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:55:29.117 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 190> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:55:30.964 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7759020, partition_space_available_1k=73331464, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1820960, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=153712, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52744, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5802524, free=2739428, buffers=272236, cached=7361448, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.13, five_min_avg=2.31, fifteen_min_avg=1.19, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:55:48.402 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 191> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:56:07.661 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 191> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:56:23.824 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:56:25.915 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:56:27.539 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 191> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:56:30.949 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7770200, partition_space_available_1k=73320284, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1858064, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=157764, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52768, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5867164, free=2666520, buffers=273536, cached=7368416, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.62, five_min_avg=2.29, fifteen_min_avg=1.25, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:56:47.301 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 192> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:57:07.113 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 192> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:57:23.826 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:57:26.050 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:57:26.469 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 192> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:57:31.079 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7773852, partition_space_available_1k=73316632, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1886824, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=162340, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52776, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5909968, free=2626256, buffers=274620, cached=7364792, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.68, five_min_avg=2.38, fifteen_min_avg=1.35, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:57:46.953 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 193> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:58:06.828 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 193> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:58:23.869 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:58:26.256 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:58:26.903 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 193> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:58:31.282 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7788896, partition_space_available_1k=73301588, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1898784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166468, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52792, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5927944, free=2591040, buffers=275312, cached=7381340, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.53, five_min_avg=2.7, fifteen_min_avg=1.53, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:58:46.995 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 194> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:59:06.509 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 194> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:59:23.857 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 00:59:26.160 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 00:59:26.296 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 194> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 00:59:31.185 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7792228, partition_space_available_1k=73298256, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1920824, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=170696, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52800, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5944504, free=2565936, buffers=275968, cached=7389228, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.67, five_min_avg=2.88, fifteen_min_avg=1.67, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 00:59:46.229 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 195> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:00:06.623 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 195> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:00:23.925 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:00:25.948 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:00:26.997 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 195> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:00:30.987 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7798804, partition_space_available_1k=73291680, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1948888, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=174796, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52840, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5980812, free=2523588, buffers=276348, cached=7394888, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.86, five_min_avg=2.81, fifteen_min_avg=1.71, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:00:46.437 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 196> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:01:07.286 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 196> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:01:23.919 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:01:25.929 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:01:27.474 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 196> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:01:30.954 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7805180, partition_space_available_1k=73285304, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1984712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=178868, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52860, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6011596, free=2480480, buffers=276672, cached=7406888, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=2.99, fifteen_min_avg=1.85, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:01:46.989 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 197> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:02:07.519 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 197> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:02:23.868 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:02:25.917 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:02:27.609 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 197> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:02:30.943 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7811652, partition_space_available_1k=73278832, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2020708, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183456, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52860, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6071384, free=2407968, buffers=276892, cached=7419392, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.43, five_min_avg=3.06, fifteen_min_avg=1.94, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:02:47.178 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 198> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:03:06.721 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 198> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:03:23.860 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:03:25.887 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:03:27.261 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 198> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:03:30.913 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7822836, partition_space_available_1k=73267648, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2077500, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187476, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52868, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6136904, free=2329952, buffers=277124, cached=7431656, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.2, fifteen_min_avg=2.06, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:03:47.055 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 199> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:04:06.912 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 199> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:04:23.887 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:04:26.141 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:04:27.824 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 199> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:04:31.165 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7833508, partition_space_available_1k=73256976, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2113212, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=191504, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52880, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6188176, free=2267444, buffers=277280, cached=7442736, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.8, five_min_avg=3.32, fifteen_min_avg=2.18, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:04:47.722 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 200> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:05:07.645 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 200> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:05:23.909 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:05:25.852 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:05:28.441 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 200> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:05:30.888 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7843280, partition_space_available_1k=73247204, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2144008, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=195556, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52892, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6205180, free=2243884, buffers=277364, cached=7449208, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.9, five_min_avg=3.19, fifteen_min_avg=2.21, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:05:48.292 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 201> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:06:07.763 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 201> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:06:23.909 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:06:25.884 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:06:27.866 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 201> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:06:30.910 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7854612, partition_space_available_1k=73235872, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2161136, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=199608, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52892, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6219564, free=2224604, buffers=277784, cached=7453684, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.91, five_min_avg=3.12, fifteen_min_avg=2.24, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:06:48.195 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 202> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:07:08.287 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 202> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:07:23.875 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:07:25.985 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:07:28.135 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 202> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:07:31.024 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7866180, partition_space_available_1k=73224304, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2167960, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=200936, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52900, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6236712, free=2195352, buffers=277848, cached=7465724, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.84, five_min_avg=3.28, fifteen_min_avg=2.35, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:07:48.440 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 203> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:08:08.499 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 203> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:08:23.913 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:08:25.938 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:08:28.347 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 203> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:08:30.963 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7867544, partition_space_available_1k=73222940, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2174396, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=205004, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52904, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6238392, free=2181300, buffers=277896, cached=7478048, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.55, five_min_avg=3.32, fifteen_min_avg=2.42, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:08:48.027 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 204> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:09:08.787 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 204> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:09:23.929 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:09:25.962 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:09:28.775 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 204> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:09:30.989 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7873416, partition_space_available_1k=73217068, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2178908, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=209032, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52908, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6255032, free=2152976, buffers=277916, cached=7489712, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.73, five_min_avg=3.43, fifteen_min_avg=2.52, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:09:48.211 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 205> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:10:07.743 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 205> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:10:23.976 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:10:26.268 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:10:28.814 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 205> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:10:31.309 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7893360, partition_space_available_1k=73197124, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2187524, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=213200, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52920, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6270416, free=2129156, buffers=277960, cached=7498104, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=3.4, fifteen_min_avg=2.57, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:10:48.693 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 206> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:11:08.313 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 206> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:11:23.899 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:11:25.946 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:11:29.241 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 206> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:11:30.976 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7903096, partition_space_available_1k=73187388, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2204876, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=217296, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52924, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6289948, free=2099360, buffers=277976, cached=7508352, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.14, five_min_avg=3.61, fifteen_min_avg=2.69, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:11:49.647 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 207> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:12:09.163 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 207> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:12:23.930 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:12:26.039 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:12:29.387 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 207> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:12:31.070 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7913228, partition_space_available_1k=73177256, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2210664, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=221268, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52924, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6301228, free=2077684, buffers=278004, cached=7518720, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.1, five_min_avg=3.45, fifteen_min_avg=2.7, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:12:49.824 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 208> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:13:09.604 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 208> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:13:23.895 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:13:25.900 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:13:29.635 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 208> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:13:30.924 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7909892, partition_space_available_1k=73180592, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2196472, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=225764, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52932, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6303876, free=2080304, buffers=278020, cached=7513436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.92, five_min_avg=3.34, fifteen_min_avg=2.71, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:13:49.173 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 209> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:14:09.455 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 209> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:14:23.915 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:14:26.179 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:14:29.485 7fdbb0155048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 209> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:14:31.211 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7919448, partition_space_available_1k=73171036, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2193600, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=229936, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52936, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6292112, free=2092140, buffers=278024, cached=7513360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.72, five_min_avg=3.22, fifteen_min_avg=2.71, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:14:38.695 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 210> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:14:47.097 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 210> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
02/20/2025 01:15:23.900 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
02/20/2025 01:15:25.939 7fdbb5ae7df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
02/20/2025 01:15:30.972 7fdbb5ae7df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2748-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7928600, partition_space_available_1k=73161884, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2195784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=234048, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52944, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6298860, free=2075996, buffers=278024, cached=7522756, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.49, five_min_avg=3.07, fifteen_min_avg=2.69, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
02/20/2025 01:15:37.385 7fdbb0155348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2748-2.: 211> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map