01/06/2025 02:04:49.523 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
01/06/2025 02:04:49.523 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
01/06/2025 02:04:49.523 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
01/06/2025 02:04:49.529 7fabd400b648 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
01/06/2025 02:04:49.530 7fabd400b648 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
01/06/2025 02:04:49.530 7fabd400be48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
01/06/2025 02:04:49.531 7fabd400be48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2536-1., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
01/06/2025 02:04:49.531 7fabd400be48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
01/06/2025 02:04:49.531 7fabd400be48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
01/06/2025 02:04:49.532 7fabd400be48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.532 7fabd400be48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
01/06/2025 02:04:49.532 7fabd400be48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.532 7fabd400be48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.532 7fabd400be48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip =   collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
01/06/2025 02:04:49.532 7fabd400be48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.640 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 36330155036310059917463693139331186228147904501317056564448956746523495360635, 'start': '1736129087000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 35273598805279987941673276543639773626784704425168333862316989596093807667307, 'start': '1736129087000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 6324768276044449154450854613830986599534546395328518848470388704567681395241, 'start': '1736129088000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
01/06/2025 02:04:49.759 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.760 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-06 01:11:23.802032", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7305-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 8  num_core_per_socket = 8  num_thread_per_core = 1  >>  running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7  installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7  >>  
01/06/2025 02:04:49.760 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.760 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2536-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1736129089760658, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
01/06/2025 02:04:49.761 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2536-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1736129089760965, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1736129087000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1736129087000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1736129088000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-06 01:11:23.802032", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7305-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
01/06/2025 02:04:49.763 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.763 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
01/06/2025 02:04:49.763 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.763 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:49.763 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1736129087000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1736129087000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1736129088000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-06 01:11:23.802032", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7305-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
01/06/2025 02:04:49.763 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:50.032 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:04:50.331 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-2536-1., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
01/06/2025 02:04:50.333 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:50.334 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  database_usage = [ <<  disk_space_used_1k = 14128860  disk_space_available_1k = 66961624  config_db_size_1k = 108  >>,  ]  >>  
01/06/2025 02:04:50.334 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:50.695 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
01/06/2025 02:04:50.695 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2536-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1736129090695726, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
01/06/2025 02:04:50.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:50.697 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
01/06/2025 02:04:50.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:51.693 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:04:52.821 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
01/06/2025 02:04:53.531 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:04:53.532 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:04:53.532 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:04:53.532 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:53.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:53.534 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:04:53.534 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:53.715 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
01/06/2025 02:04:54.598 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
01/06/2025 02:04:55.431 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
01/06/2025 02:04:56.277 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
01/06/2025 02:04:57.155 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
01/06/2025 02:04:57.533 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:04:57.535 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:04:57.536 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:57.536 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:04:57.536 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:57.536 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:04:57.536 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:04:57.536 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:01.535 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:01.536 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:01.536 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:01.536 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:01.537 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:01.538 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:02.181 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2536-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1736129102181037, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1736129087000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1736129087000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1736129088000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-06 01:11:23.802032", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7305-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
01/06/2025 02:05:02.181 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14537028, partition_space_available_1k=66553456, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1220664, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51188, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51520, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=4770252, free=13045416, buffers=582364, cached=14267088, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.45, five_min_avg=1.35, fifteen_min_avg=0.99, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:05:02.181 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:02.182 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1736129087000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1736129087000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1736129088000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-06 01:11:23.802032", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7305-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
01/06/2025 02:05:02.182 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:02.182 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:02.182 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 14537028  partition_space_available_1k = 66553456  percentage_partition_space_used = 18  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1220664  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 51188  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51520  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 4770252  free = 13045416  buffers = 582364  cached = 14267088  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.45  five_min_avg = 1.35  fifteen_min_avg = 0.99  cpu_share = 0  node_type = config-database-node  >>  >>  
01/06/2025 02:05:02.182 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:05.537 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:05.538 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:05.538 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:05.538 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:05.538 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:05.538 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:05.538 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:05.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:09.538 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:09.539 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:09.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:09.539 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:09.539 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:09.540 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:13.542 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:13.543 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:13.543 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:13.543 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:13.543 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:13.544 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:17.546 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:17.558 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:17.562 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:17.563 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:17.563 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:17.563 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:17.563 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:17.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:17.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:17.564 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:17.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:17.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:17.564 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:17.565 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:21.559 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:21.560 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:21.560 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:21.560 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:21.561 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:21.562 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:21.562 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:21.562 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:25.562 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:25.563 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:25.563 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:25.563 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:25.563 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:25.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:25.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:25.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:25.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:25.564 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:25.564 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:25.565 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:25.565 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:25.565 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:29.567 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:29.568 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:29.568 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:29.568 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:29.569 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:29.570 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:33.572 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:33.573 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:33.573 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:33.573 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:33.573 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:33.573 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:33.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:37.574 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:37.575 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:37.575 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:37.575 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:37.575 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:37.576 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:41.579 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:41.580 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:41.580 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:41.580 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:41.580 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:41.580 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:41.580 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:41.581 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:45.582 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:45.583 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:45.583 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:45.583 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:45.583 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:45.584 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:49.586 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:49.587 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:49.588 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:49.588 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:49.588 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:49.588 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:49.588 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:49.589 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:49.984 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:05:50.275 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:50.276 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  database_usage = [ <<  disk_space_used_1k = 15398708  disk_space_available_1k = 65691776  config_db_size_1k = 1460  >>,  ]  >>  
01/06/2025 02:05:50.276 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:50.779 7fabd941adf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2536-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1736129150779826, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.31:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
01/06/2025 02:05:50.781 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:50.781 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:50.781 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:51.604 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:05:53.588 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:53.589 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:53.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:53.591 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:53.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:53.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:53.591 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:53.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:56.627 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15398720, partition_space_available_1k=65691764, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1895448, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51196, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51632, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=5361432, free=11511200, buffers=644664, cached=15147824, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.23, five_min_avg=1.59, fifteen_min_avg=1.1, cpu_share=2.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:05:56.627 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:56.627 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 15398720  partition_space_available_1k = 65691764  percentage_partition_space_used = 19  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1895448  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 51196  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51632  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 5361432  free = 11511200  buffers = 644664  cached = 15147824  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.23  five_min_avg = 1.59  fifteen_min_avg = 1.1  cpu_share = 2.75  node_type = config-database-node  >>  >>  
01/06/2025 02:05:56.627 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:57.590 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:05:57.591 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:05:57.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:05:57.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:57.591 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:05:57.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:05:57.591 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:05:57.592 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:01.594 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:01.595 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:01.596 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:01.597 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:01.597 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:01.597 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:01.597 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:01.597 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:05.598 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:05.601 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:05.601 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:05.601 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:05.601 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:05.601 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:05.601 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:05.602 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:09.602 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:09.604 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:09.604 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:09.605 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:09.605 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:09.605 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:09.605 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:09.606 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:09.606 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:09.606 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:09.606 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:09.606 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:09.607 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:09.607 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:13.606 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:13.607 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:13.607 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:13.607 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:13.608 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:13.609 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:13.609 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:13.609 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:17.610 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:17.611 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:17.611 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:17.612 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:17.612 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:17.612 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:17.612 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:17.613 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:21.613 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:21.614 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:21.614 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:21.614 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:21.614 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:21.614 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:21.615 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:25.614 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:25.615 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:25.616 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:25.616 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:25.616 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:25.616 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:25.616 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:25.617 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:29.618 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:29.619 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:29.619 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:29.619 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:29.620 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:33.622 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:33.623 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:33.624 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:33.624 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:33.624 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:33.624 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:33.624 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:33.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:37.624 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:37.625 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:37.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:37.625 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:37.625 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:37.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:41.626 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:41.627 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:41.627 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:41.627 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:41.627 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:41.627 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:41.627 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:41.628 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:45.628 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:45.629 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:45.629 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:45.629 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:45.629 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:45.629 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:45.629 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:45.630 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:49.630 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:49.631 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:49.632 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:49.633 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:49.633 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:49.633 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:49.633 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:50.015 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:06:50.342 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:50.342 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  database_usage = [ <<  disk_space_used_1k = 15703544  disk_space_available_1k = 65386940  config_db_size_1k = 1576  >>,  ]  >>  
01/06/2025 02:06:50.343 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:51.861 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:06:53.632 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:53.634 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:53.634 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:53.634 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:53.634 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:53.634 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:53.634 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:53.635 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:56.884 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15704276, partition_space_available_1k=65386208, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1923632, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78980, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52288, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6129372, free=10243148, buffers=672452, cached=15620148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.55, five_min_avg=1.77, fifteen_min_avg=1.19, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:06:56.884 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:56.884 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 15704276  partition_space_available_1k = 65386208  percentage_partition_space_used = 19  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1923632  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 78980  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 52288  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 6129372  free = 10243148  buffers = 672452  cached = 15620148  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.55  five_min_avg = 1.77  fifteen_min_avg = 1.19  cpu_share = 3.62  node_type = config-database-node  >>  >>  
01/06/2025 02:06:56.885 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:57.635 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:06:57.636 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:06:57.637 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:57.637 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:06:57.637 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:57.637 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:06:57.637 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:06:57.637 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:01.638 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:01.639 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:01.640 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:01.640 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:01.640 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:01.640 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:01.640 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:01.641 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:01.641 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:01.641 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:01.641 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:01.641 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:01.641 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:01.642 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:05.643 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:05.644 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:05.644 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:05.644 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:05.644 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:05.644 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:05.645 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:09.646 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:09.647 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:09.647 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:09.647 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:09.648 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:09.649 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:13.650 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:13.651 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:13.652 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:13.652 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:13.652 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:13.652 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:13.652 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:13.653 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:17.654 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:17.655 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:17.655 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:17.655 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:17.655 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:17.655 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:17.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:21.656 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:21.658 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:21.658 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:21.658 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:21.658 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:21.658 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:21.658 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:21.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:25.658 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:25.659 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:25.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:25.659 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:25.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:25.661 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:25.661 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:29.660 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:29.661 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:29.661 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:29.661 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:29.661 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:29.662 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:33.665 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:33.666 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:33.666 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:33.667 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:33.667 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:33.667 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:33.667 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:33.668 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:37.670 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:37.671 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:37.671 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:37.671 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:37.671 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:37.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:41.672 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:41.673 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:41.673 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:41.673 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:41.673 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:41.674 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:45.673 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:45.674 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:45.675 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:45.676 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:45.676 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:45.676 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:45.676 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:49.677 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:49.678 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:49.678 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:49.678 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:49.678 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:49.678 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:49.678 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:49.679 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:49.998 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:07:50.332 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:50.332 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  database_usage = [ <<  disk_space_used_1k = 16208812  disk_space_available_1k = 64881672  config_db_size_1k = 1648  >>,  ]  >>  
01/06/2025 02:07:50.333 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:51.682 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:07:53.678 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:53.680 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:53.680 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:53.680 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:53.680 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:53.680 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:53.680 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:53.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:56.706 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16208824, partition_space_available_1k=64881660, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2019300, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=79924, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52416, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6388908, free=9521412, buffers=698308, cached=16056492, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.92, five_min_avg=1.73, fifteen_min_avg=1.21, cpu_share=2.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:07:56.706 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:56.706 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 16208824  partition_space_available_1k = 64881660  percentage_partition_space_used = 20  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 2019300  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 79924  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 52416  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 6388908  free = 9521412  buffers = 698308  cached = 16056492  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.92  five_min_avg = 1.73  fifteen_min_avg = 1.21  cpu_share = 2.38  node_type = config-database-node  >>  >>  
01/06/2025 02:07:56.706 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:57.680 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:07:57.681 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:07:57.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:07:57.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:57.681 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:07:57.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:07:57.681 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:07:57.682 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:01.685 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:01.686 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:01.686 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:01.686 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:01.686 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:01.686 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:01.686 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:01.687 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:05.686 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:05.687 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:05.688 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:05.689 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:05.689 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:05.689 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:09.690 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:09.691 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:09.691 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:09.691 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:09.692 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:09.693 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:09.693 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:09.693 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:13.692 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:13.693 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:13.693 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:13.693 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:13.693 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:13.693 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:13.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:17.694 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:17.695 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:17.696 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:17.696 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:17.696 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:17.696 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:17.696 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:17.697 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:21.699 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:21.700 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:21.700 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:21.700 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:21.700 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:21.700 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:21.700 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:21.701 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:25.702 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:25.703 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:25.703 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:25.703 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:25.703 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:25.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:29.706 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:29.707 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:29.707 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:29.707 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:29.708 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:29.709 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:33.710 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:33.711 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:33.711 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:33.711 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:33.712 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:33.713 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:33.713 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:33.713 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:37.712 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:37.713 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:37.713 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:37.713 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:37.714 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:37.715 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:41.717 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:41.718 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:41.718 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:41.718 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:41.718 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:41.718 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:41.718 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:41.719 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:45.721 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:45.723 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:45.724 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:45.724 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:45.724 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:45.724 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:45.724 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:45.724 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:49.724 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:49.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:49.726 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:49.726 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:49.726 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:49.726 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:49.726 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:50.010 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:08:50.344 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:50.344 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  database_usage = [ <<  disk_space_used_1k = 16222884  disk_space_available_1k = 64867600  config_db_size_1k = 1648  >>,  ]  >>  
01/06/2025 02:08:50.344 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:51.680 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:08:53.725 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:53.726 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:53.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:53.728 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:53.728 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:53.728 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:53.728 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:53.728 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:56.703 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16222772, partition_space_available_1k=64867712, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2014440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80324, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52560, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7733328, free=8154300, buffers=700632, cached=16076860, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.85, five_min_avg=1.76, fifteen_min_avg=1.25, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:08:56.703 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:56.704 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 16222772  partition_space_available_1k = 64867712  percentage_partition_space_used = 20  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 2014440  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 80324  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 52560  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 7733328  free = 8154300  buffers = 700632  cached = 16076860  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.85  five_min_avg = 1.76  fifteen_min_avg = 1.25  cpu_share = 2.62  node_type = config-database-node  >>  >>  
01/06/2025 02:08:56.704 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:57.727 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:57.729 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:08:57.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:57.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:08:57.730 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:08:57.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:01.729 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:09:01.730 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:09:01.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:09:01.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:01.730 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:09:01.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:01.730 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:09:01.731 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:05.733 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:09:05.734 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:09:05.734 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:09:05.734 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:05.734 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:09:05.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:09.735 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:09:09.736 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
01/06/2025 02:09:09.736 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:09:09.736 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:09.736 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:09:09.736 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:09.736 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1736129089531659  collector_name =   collector_ip = 10.0.0.31:8086  collector_list = [ 10.0.0.31:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
01/06/2025 02:09:09.737 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
01/06/2025 02:09:13.738 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
01/06/2025 02:09:13.739 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
01/06/2025 02:09:13.739 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
01/06/2025 02:09:13.739 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-2536-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.31:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
01/06/2025 02:09:13.740 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
01/06/2025 02:09:22.179 7fabd3bda348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
01/06/2025 02:09:22.179 7fabd3bda348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
01/06/2025 02:09:22.180 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
01/06/2025 02:09:22.180 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
01/06/2025 02:09:22.180 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 5> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
01/06/2025 02:09:22.180 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 144> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
01/06/2025 02:09:22.180 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 67> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
01/06/2025 02:09:22.180 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
01/06/2025 02:09:22.181 7fabd2d22048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
01/06/2025 02:09:22.182 7fabd2d22048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
01/06/2025 02:09:33.025 7fabd3bda348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439
01/06/2025 02:09:33.027 7fabd3bda348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
01/06/2025 02:09:33.029 7fabd2d22048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
01/06/2025 02:09:33.029 7fabd2d22048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
01/06/2025 02:09:33.031 7fabd2d22048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 5> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
01/06/2025 02:09:33.031 7fabd2d22048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 145> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
01/06/2025 02:09:33.032 7fabd2d22048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 68> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
01/06/2025 02:09:33.034 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
01/06/2025 02:09:33.035 7fabd2d22348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
01/06/2025 02:09:50.088 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:09:51.970 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:09:56.996 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16803780, partition_space_available_1k=64286704, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2015912, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=161424, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52824, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8935492, free=5678216, buffers=724844, cached=17326568, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=2.19, fifteen_min_avg=1.43, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:10:50.119 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:10:52.323 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:10:57.348 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17578988, partition_space_available_1k=63511496, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2015808, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=174264, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44524, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8831888, free=20450932, buffers=735172, cached=2647128, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.32, five_min_avg=2.39, fifteen_min_avg=1.55, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:11:25.729 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 147> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
01/06/2025 02:11:27.647 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 147> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
01/06/2025 02:11:50.077 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:11:52.313 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:11:57.334 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18065916, partition_space_available_1k=63024568, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2018704, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=181472, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44584, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9544480, free=18476364, buffers=748408, cached=3895868, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.48, five_min_avg=2.27, fifteen_min_avg=1.56, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:12:50.113 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:12:52.053 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:12:57.079 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19469468, partition_space_available_1k=61621016, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2026888, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=182464, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44616, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9971448, free=17166540, buffers=782268, cached=4744864, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.46, five_min_avg=2.32, fifteen_min_avg=1.62, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:13:50.111 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:13:52.032 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:13:57.060 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19500204, partition_space_available_1k=61590280, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2017964, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185068, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44620, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10141164, free=16927960, buffers=782992, cached=4813004, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=2.44, fifteen_min_avg=1.71, cpu_share=4.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:14:50.199 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:14:52.515 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:14:57.545 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19562876, partition_space_available_1k=61527608, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2027356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188812, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44636, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10915400, free=16098668, buffers=783100, cached=4867952, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=2.6, fifteen_min_avg=1.81, cpu_share=5.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:15:50.127 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:15:52.092 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:15:57.113 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19581184, partition_space_available_1k=61509300, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2038536, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=196100, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44636, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10390036, free=16598020, buffers=783200, cached=4893864, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.05, five_min_avg=3.73, fifteen_min_avg=2.27, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:16:50.133 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:16:52.052 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:16:57.076 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19603288, partition_space_available_1k=61487196, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2030796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=198488, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44644, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10885716, free=16078744, buffers=783400, cached=4917260, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=3.56, fifteen_min_avg=2.3, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:17:50.121 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:17:52.050 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:17:57.075 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19627940, partition_space_available_1k=61462544, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2032284, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=200244, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44652, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11394412, free=15544844, buffers=783440, cached=4942424, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=3.45, fifteen_min_avg=2.34, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:18:50.120 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
01/06/2025 02:18:51.995 7fabd941adf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
01/06/2025 02:18:57.019 7fabd941adf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2536-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19775056, partition_space_available_1k=61315428, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2034712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=201104, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44656, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10306752, free=16468060, buffers=785912, cached=5104396, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.46, five_min_avg=3.19, fifteen_min_avg=2.32, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
01/06/2025 02:19:03.178 7fabd2d22348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-2536-1.: 155> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map