05/05/2025 01:53:05.515 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
05/05/2025 01:53:05.515 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
05/05/2025 01:53:05.515 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
05/05/2025 01:53:05.523 7fe6723166d8 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
05/05/2025 01:53:05.523 7fe6723166d8 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
05/05/2025 01:53:05.524 7fe67231d048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3264-1., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.525 7fe67231d048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.526 7fe67231d048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip =   collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
05/05/2025 01:53:05.526 7fe67231d048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.638 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 26418704838366982260771718387610696079004494063376046184026032582782422759359, 'start': '1746409983000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 89795852633480509743807415036692084581886303219810782004228930684086003328606, 'start': '1746409983000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 4701170217451953536944223700521991643861304137642266254011269882879597650652, 'start': '1746409984000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
05/05/2025 01:53:05.758 7fe67ae2da20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.759 7fe67ae2da20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-05 00:40:42.530824", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9067-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 8  num_core_per_socket = 8  num_thread_per_core = 1  >>  running_package_version = nightly-nightly.el7  installed_package_version = nightly-nightly.el7  >>  
05/05/2025 01:53:05.759 7fe67ae2da20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.760 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3264-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1746409985760104, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
05/05/2025 01:53:05.760 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3264-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1746409985760401, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1746409983000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1746409983000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1746409984000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-05 00:40:42.530824", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9067-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
05/05/2025 01:53:05.762 7fe674741438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.762 7fe674741438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/05/2025 01:53:05.762 7fe674741438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.762 7fe674741438 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:05.762 7fe674741438 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746409983000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746409983000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746409984000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-05 00:40:42.530824", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9067-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
05/05/2025 01:53:05.762 7fe674741438 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:06.026 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:53:06.296 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3264-1., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
05/05/2025 01:53:06.298 7fe67227e2e8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:06.298 7fe67227e2e8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  database_usage = [ <<  disk_space_used_1k = 13417780  disk_space_available_1k = 67672704  config_db_size_1k = 108  >>,  ]  >>  
05/05/2025 01:53:06.298 7fe67227e2e8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:06.671 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
05/05/2025 01:53:06.671 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3264-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1746409986671300, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
05/05/2025 01:53:06.672 7fe6710cb6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:06.672 7fe6710cb6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/05/2025 01:53:06.672 7fe6710cb6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:07.555 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:53:08.522 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
05/05/2025 01:53:09.315 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
05/05/2025 01:53:09.525 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:09.527 7fe674959828 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:09.527 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:09.527 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:09.527 7fe674959828 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:09.527 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:09.527 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:09.528 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:09.528 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:09.528 7fe674959828 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:09.528 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:09.528 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:09.528 7fe674959828 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:09.529 7fe674959828 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:10.080 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
05/05/2025 01:53:10.818 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
05/05/2025 01:53:11.576 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
05/05/2025 01:53:12.313 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
05/05/2025 01:53:13.528 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:13.529 7fe67052e6d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:13.529 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:13.529 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:13.529 7fe67052e6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:13.529 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:13.529 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:13.530 7fe67052e6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.334 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3264-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1746409997333955, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1746409983000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1746409983000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1746409984000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-05 00:40:42.530824", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9067-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
05/05/2025 01:53:17.334 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13667540, partition_space_available_1k=67422944, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1220852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50188, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51264, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=4707116, free=13633796, buffers=613416, cached=13710792, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.8, five_min_avg=1.31, fifteen_min_avg=0.85, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 01:53:17.334 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.334 7fe6745c28d0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746409983000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746409983000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1746409984000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-05 00:40:42.530824", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9067-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
05/05/2025 01:53:17.334 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.334 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.335 7fe6745c28d0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 13667540  partition_space_available_1k = 67422944  percentage_partition_space_used = 17  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1220852  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 50188  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51264  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 4707116  free = 13633796  buffers = 613416  cached = 13710792  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.8  five_min_avg = 1.31  fifteen_min_avg = 0.85  cpu_share = 0  node_type = config-database-node  >>  >>  
05/05/2025 01:53:17.335 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.530 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:17.531 7fe6745c28d0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:17.531 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:17.531 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:17.532 7fe6745c28d0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:21.533 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:21.534 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:21.535 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:21.535 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:21.535 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:21.535 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:21.535 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:25.537 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:25.538 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:25.539 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:25.539 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:25.539 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:25.539 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:25.539 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:25.539 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:29.542 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:29.543 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:29.543 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:29.543 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:29.543 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:29.543 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:29.543 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:29.544 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:33.545 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:33.546 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:33.546 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:33.546 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:33.547 7fe67229b048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:37.548 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:37.549 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:37.550 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:37.550 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:37.550 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:37.550 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:37.550 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:37.550 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:41.553 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:41.553 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:41.554 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:41.554 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:41.554 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:41.554 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:41.554 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:41.554 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:41.555 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:41.555 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:41.555 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:41.555 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:41.555 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:41.555 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:45.557 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:45.558 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:45.559 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:45.559 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:45.559 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:45.559 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:45.559 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:45.559 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:49.562 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:49.645 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:49.645 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:49.646 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:49.646 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:49.646 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:49.646 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:49.646 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:49.647 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:49.647 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:49.647 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:49.647 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:49.647 7fe67475d6d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:49.648 7fe67475d6d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:53.648 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:53.649 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:53.650 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:53.650 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:53.650 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:53.650 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:53.650 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:53.650 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:57.653 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:53:57.654 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:53:57.655 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:53:57.655 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:57.655 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:53:57.655 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:53:57.655 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:53:57.656 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:01.659 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:01.660 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:01.661 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:01.661 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:01.661 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:01.661 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:01.661 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:01.661 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:05.661 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:05.662 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:05.663 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:05.664 7fe6747416d8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:05.664 7fe6747416d8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:05.968 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:54:06.255 7fe679d8cf60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:06.255 7fe679d8cf60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  database_usage = [ <<  disk_space_used_1k = 14411092  disk_space_available_1k = 66679392  config_db_size_1k = 1648  >>,  ]  >>  
05/05/2025 01:54:06.255 7fe679d8cf60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:06.805 7fe674bb0188 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3264-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1746410046805722, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.130:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
05/05/2025 01:54:06.807 7fe6745560f0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:06.807 7fe6745560f0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:06.807 7fe6745560f0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:07.636 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:54:09.664 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:09.666 7fe67050f588 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:09.666 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:09.667 7fe67050f588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:09.668 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:09.668 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:09.668 7fe67050f588 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:09.668 7fe67050f588 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:12.658 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14425956, partition_space_available_1k=66664528, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1924440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=79032, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51676, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6186624, free=11118372, buffers=667284, cached=14692840, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=1.88, fifteen_min_avg=1.07, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 01:54:12.658 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:12.658 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 14425956  partition_space_available_1k = 66664528  percentage_partition_space_used = 18  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1924440  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 79032  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51676  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 6186624  free = 11118372  buffers = 667284  cached = 14692840  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 3.59  five_min_avg = 1.88  fifteen_min_avg = 1.07  cpu_share = 4.0  node_type = config-database-node  >>  >>  
05/05/2025 01:54:12.658 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:13.668 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:13.669 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:13.669 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:13.669 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:13.670 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:13.671 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:13.671 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:13.671 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:17.671 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:17.672 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:17.672 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:17.672 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:17.672 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:17.672 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:17.672 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:17.673 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:21.672 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:21.673 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:21.674 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:21.675 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:21.675 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:21.675 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:21.675 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:21.675 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:25.678 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:25.679 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:25.679 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:25.679 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:25.680 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:29.681 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:29.681 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:29.682 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:29.683 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:29.683 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:33.685 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:33.686 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:33.686 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:33.687 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:33.687 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:33.687 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:33.687 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:33.687 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:33.687 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:33.688 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:33.688 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:33.688 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:33.688 7fe67223dd68 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:33.688 7fe67223dd68 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:37.687 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:37.688 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:37.689 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:37.689 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:37.689 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:37.689 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:37.689 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:37.689 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:37.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:37.690 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:37.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:37.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:37.690 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:37.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:41.689 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:41.690 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:41.691 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:41.691 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:41.691 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:41.691 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:45.692 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:45.694 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:45.694 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:45.694 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:45.694 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:45.694 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:45.694 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:45.695 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:45.695 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:45.695 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:45.695 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:45.695 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:45.696 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:45.696 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:49.698 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:49.699 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:49.699 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:49.699 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:49.699 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:49.699 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:49.699 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:49.700 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:53.699 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:53.700 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:53.701 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:53.702 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:53.702 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:53.702 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:53.702 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:57.705 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:54:57.705 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:57.706 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:54:57.707 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:57.707 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:54:57.707 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:54:57.707 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:01.709 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:01.709 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:01.710 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:01.711 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:01.711 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:01.711 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:01.711 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:05.714 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:05.715 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:05.715 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:05.715 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:05.715 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:05.715 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:05.715 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:05.716 7fe674693978 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:05.980 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:55:06.268 7fe67233d2e8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:06.268 7fe67233d2e8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  database_usage = [ <<  disk_space_used_1k = 15567404  disk_space_available_1k = 65523080  config_db_size_1k = 1612  >>,  ]  >>  
05/05/2025 01:55:06.268 7fe67233d2e8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:07.581 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:55:09.717 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:09.718 7fe675d9fac8 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:09.718 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:09.718 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:09.719 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:09.720 7fe675d9fac8 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:09.720 7fe675d9fac8 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:12.681 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15670532, partition_space_available_1k=65419952, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1926760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80816, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51988, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7650568, free=8583764, buffers=697428, cached=15733360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.97, five_min_avg=2.0, fifteen_min_avg=1.16, cpu_share=3.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 01:55:12.681 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:12.681 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 15670532  partition_space_available_1k = 65419952  percentage_partition_space_used = 19  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1926760  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 80816  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51988  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665120  used = 7650568  free = 8583764  buffers = 697428  cached = 15733360  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.97  five_min_avg = 2.0  fifteen_min_avg = 1.16  cpu_share = 3.25  node_type = config-database-node  >>  >>  
05/05/2025 01:55:12.681 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:13.720 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:13.721 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:13.721 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:13.721 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:13.721 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:13.721 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:13.721 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:13.722 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:17.726 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:17.727 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:17.727 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:17.727 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:17.727 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:17.727 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:17.727 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:17.728 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:21.729 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:21.730 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:21.730 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:21.730 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:21.730 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:21.730 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:21.731 7fe6710dfe10 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:25.734 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:25.735 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/05/2025 01:55:25.736 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:25.736 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:25.736 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:25.736 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:25.736 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/05/2025 01:55:25.736 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/05/2025 01:55:25.737 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:25.737 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/05/2025 01:55:25.737 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:25.737 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:25.737 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1746409985525436  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/05/2025 01:55:25.737 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/05/2025 01:55:29.737 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/05/2025 01:55:29.737 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
05/05/2025 01:55:29.738 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/05/2025 01:55:29.738 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:29.739 7fe67468ba20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3264-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/05/2025 01:55:29.739 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/05/2025 01:55:29.739 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
05/05/2025 01:55:29.739 7fe67468ba20 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
05/05/2025 01:55:29.740 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
05/05/2025 01:55:29.740 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
05/05/2025 01:55:29.740 7fe67468ba20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
05/05/2025 01:55:35.682 7fe672313198 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
05/05/2025 01:55:35.683 7fe672313198 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
05/05/2025 01:55:35.683 7fe674524eb8 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
05/05/2025 01:55:35.683 7fe674524eb8 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
05/05/2025 01:55:35.684 7fe674524eb8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 3> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
05/05/2025 01:55:35.684 7fe674524eb8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 82> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
05/05/2025 01:55:35.684 7fe674524eb8 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 37> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/05/2025 01:55:35.684 7fe674524eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
05/05/2025 01:55:35.684 7fe67224a048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
05/05/2025 01:55:35.685 7fe67224a048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
05/05/2025 01:55:45.667 7fe672313198 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438
05/05/2025 01:55:45.669 7fe672313198 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
05/05/2025 01:55:45.670 7fe67050f390 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
05/05/2025 01:55:45.670 7fe67050f390 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
05/05/2025 01:55:45.670 7fe67050f390 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 3> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
05/05/2025 01:55:45.671 7fe67050f390 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 83> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
05/05/2025 01:55:45.671 7fe67050f390 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 38> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/05/2025 01:55:45.671 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
05/05/2025 01:55:45.672 7fe672316eb8 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
05/05/2025 01:56:06.022 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:56:07.647 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:56:13.647 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17389580, partition_space_available_1k=63700904, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2025648, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=157752, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52000, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8957092, free=15247736, buffers=741108, cached=7719184, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=2.28, fifteen_min_avg=1.31, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 01:57:06.030 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:57:07.780 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:57:13.628 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17416168, partition_space_available_1k=63674316, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2015420, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=162788, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42400, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9578840, free=19750756, buffers=745036, cached=2590488, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=2.27, fifteen_min_avg=1.37, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 01:57:20.820 7fe67b7d20f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 85> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
05/05/2025 01:57:22.781 7fe67b7d20f0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 85> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
05/05/2025 01:58:06.039 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:58:07.714 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:58:12.883 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17979648, partition_space_available_1k=63110836, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2016004, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=163028, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42780, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9676632, free=18542304, buffers=766024, cached=3680160, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=2.25, fifteen_min_avg=1.42, cpu_share=3.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 01:59:06.108 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 01:59:08.131 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 01:59:14.008 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18164820, partition_space_available_1k=62925664, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2029300, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184452, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42780, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9871280, free=18232540, buffers=770336, cached=3790964, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.98, five_min_avg=2.85, fifteen_min_avg=1.67, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 02:00:06.091 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
05/05/2025 02:00:08.031 7fe674bb0188 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
05/05/2025 02:00:13.919 7fe674bb0188 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3264-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18219000, partition_space_available_1k=62871484, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2032052, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=206012, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42788, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9911252, free=18133740, buffers=770388, cached=3849740, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.63, five_min_avg=3.41, fifteen_min_avg=1.94, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
05/05/2025 02:00:28.644 7fe6710c4d68 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3264-1.: 88> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map