09/01/2025 01:59:38.597 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/01/2025 01:59:38.597 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/01/2025 01:59:38.597 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/01/2025 01:59:38.605 7f5f030e74a0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 09/01/2025 01:59:38.606 7f5f030e74a0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 09/01/2025 01:59:38.607 7f5f030e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/01/2025 01:59:38.607 7f5f030e7860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 01:59:38.608 7f5f030e7860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/01/2025 01:59:38.608 7f5f030e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/01/2025 01:59:38.608 7f5f030e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.608 7f5f030e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:59:38.609 7f5f030e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.609 7f5f030e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.609 7f5f030e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/01/2025 01:59:38.609 7f5f030e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.899 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 2970315137873738837294689501069618496038428315884602669403487604349133437389, 'start': '1756691976000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 41571860952625363574189541715747291034278630777381198647066155802220471397141, 'start': '1756691978000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 98427404881738005355690242207659557540002537806060955137945532399724160633091, 'start': '1756691976000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/01/2025 01:59:39.019 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.020 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 09/01/2025 01:59:39.020 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.021 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3993-1.', _node_type='Database', _instance_id='0', _timestamp=1756691979021387, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:59:39.022 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3993-1.', _node_type='Database', _instance_id='0', _timestamp=1756691979022016, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691976000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691978000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691976000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:59:39.025 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.026 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:59:39.026 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.026 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.026 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691976000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691978000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691976000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:59:39.026 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.346 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 01:59:39.348 7f5f030e7900 [contrail-database-nodemgr] [DEBUG]: DBG: event_process_state:contrail-query-engine,group:contrail-query-engine,state:PROCESS_STATE_BACKOFF 09/01/2025 01:59:39.690 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/01/2025 01:59:39.692 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.692 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 16901232 disk_space_available_1k = 64189252 analytics_db_size_1k = 108 >>, ] >> 09/01/2025 01:59:39.692 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.902 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/01/2025 01:59:39.902 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3993-1.', _node_type='Database', _instance_id='0', _timestamp=1756691979902899, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:59:39.904 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:39.904 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:59:39.905 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:40.927 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 01:59:42.015 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1 09/01/2025 01:59:42.608 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:42.610 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:42.610 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:42.610 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:42.610 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:42.610 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:42.610 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:42.611 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:42.612 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.612 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:42.612 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.612 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.613 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:42.613 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.051 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3993-1.', _node_type='Database', _instance_id='0', _timestamp=1756691985051617, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691976000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_BACKOFF', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691978000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1756691976000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 01:59:45.052 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17042388, partition_space_available_1k=64048096, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1262528, cpu_share=0.0, mem_res=1252364, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52036, cpu_share=0.0, mem_res=42636, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7404912, free=7508184, buffers=665220, cached=17086804, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.49, five_min_avg=2.75, fifteen_min_avg=1.91, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 01:59:45.053 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.053 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691976000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_BACKOFF start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691978000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1756691976000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-01 00:14:10.876612", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10717-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/01/2025 01:59:45.054 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.054 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:45.054 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17042388 partition_space_available_1k = 64048096 percentage_partition_space_used = 21 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1262528 cpu_share = 0.0 mem_res = 1252364 >>contrail-database-nodemgr : << mem_virt = 52036 cpu_share = 0.0 mem_res = 42636 >> } system_mem_usage = << total = 32665120 used = 7404912 free = 7508184 buffers = 665220 cached = 17086804 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.49 five_min_avg = 2.75 fifteen_min_avg = 1.91 cpu_share = 0 node_type = database-node >> >> 09/01/2025 01:59:45.054 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.613 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:46.650 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:46.650 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:46.650 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:46.650 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:46.650 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:46.650 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:46.651 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:46.651 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.651 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:46.651 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.651 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.652 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:46.652 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.651 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:50.652 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:50.652 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:50.652 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:50.652 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.653 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.654 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:50.654 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.653 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:54.658 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:54.658 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:54.658 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:54.659 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:54.659 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:54.659 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:54.660 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:54.661 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.661 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:54.661 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.661 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.662 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:54.662 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.660 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:58.662 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:58.663 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:58.663 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:58.664 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:58.664 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:58.664 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:58.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:58.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.666 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:58.666 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.666 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.667 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 01:59:58.667 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.664 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:02.667 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:02.668 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:02.668 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:02.669 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:02.669 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:02.669 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:02.671 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:02.671 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.671 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:02.672 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.672 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.673 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:02.673 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.668 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:06.670 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:06.671 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:06.671 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:06.672 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:06.672 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:06.672 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:06.673 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:06.673 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.674 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:06.674 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.674 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.675 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:06.675 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.671 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:10.674 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:10.674 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:10.674 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:10.675 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:10.675 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:10.675 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:10.676 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:10.676 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.677 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:10.677 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.677 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.677 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:10.678 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.677 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:14.680 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:14.680 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:14.680 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:14.681 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:14.681 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:14.681 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:14.682 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:14.683 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.683 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:14.683 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.683 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.684 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:14.684 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.684 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:18.687 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:18.687 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:18.688 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:18.688 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:18.688 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:18.689 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:18.689 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:18.690 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.690 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:18.690 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.691 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.691 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:18.691 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.688 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:22.690 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:22.691 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:22.691 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:22.691 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:22.691 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:22.691 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:22.692 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:22.692 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.693 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:22.693 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.693 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.693 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:22.694 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.692 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:26.694 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:26.694 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:26.694 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:26.694 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:26.695 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:26.695 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:26.695 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:26.695 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.696 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:26.696 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.696 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.696 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:26.696 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.696 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:30.698 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:30.699 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:30.699 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:30.700 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:30.700 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:30.700 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:30.701 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:30.702 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.702 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:30.702 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.703 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.703 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:30.703 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.700 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:34.703 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:34.703 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:34.703 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:34.704 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:34.704 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:34.704 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:34.705 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:34.705 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.705 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:34.705 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.705 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.706 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:34.706 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.707 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:38.709 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:38.710 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:38.710 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:38.710 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:38.711 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:38.711 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:38.712 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:38.712 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.713 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:38.713 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.713 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.714 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:38.714 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:39.465 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:00:40.071 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:40.071 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 18063352 disk_space_available_1k = 63027132 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:00:40.071 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:40.293 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:00:40.294 7f5f05f7b940 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3993-1.', _node_type='Database', _instance_id='0', _timestamp=1756692040294358, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.56:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/01/2025 02:00:40.297 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:40.298 7f5f031e7220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:40.298 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:41.300 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:00:42.711 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:42.714 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:42.715 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:42.715 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:42.716 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:42.716 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:42.716 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:42.717 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:42.718 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.718 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:42.718 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.719 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.720 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:42.720 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:44.360 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17895656, partition_space_available_1k=63194828, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1289796, cpu_share=0.14, mem_res=1279476, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52284, cpu_share=0.03, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7625544, free=6457964, buffers=675240, cached=17906372, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=2.77, fifteen_min_avg=1.97, cpu_share=3.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:00:44.361 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:44.361 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 17895656 partition_space_available_1k = 63194828 percentage_partition_space_used = 22 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1289796 cpu_share = 0.14 mem_res = 1279476 >>contrail-database-nodemgr : << mem_virt = 52284 cpu_share = 0.03 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 7625544 free = 6457964 buffers = 675240 cached = 17906372 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.58 five_min_avg = 2.77 fifteen_min_avg = 1.97 cpu_share = 3.75 node_type = database-node >> >> 09/01/2025 02:00:44.362 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.718 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:46.720 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:46.720 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:46.720 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:46.721 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:46.721 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:46.721 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:46.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:46.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.722 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:46.723 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.723 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.723 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:46.723 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.720 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:50.721 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:50.721 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:50.721 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.722 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:50.723 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.725 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:54.726 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:54.726 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:54.726 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:54.726 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:54.726 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:54.726 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:54.727 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.728 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:58.731 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:58.731 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:58.731 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:58.732 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:58.732 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:58.732 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:58.733 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:58.733 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.733 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:58.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.734 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:00:58.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.733 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:02.734 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:02.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:02.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:02.734 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:02.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:02.734 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:02.735 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:02.736 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.737 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:02.737 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.737 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.737 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:02.737 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.738 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:06.740 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:06.741 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:06.741 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:06.741 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:06.742 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:06.742 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:06.743 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:06.743 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.743 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:06.744 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.744 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.744 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:06.744 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.740 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:10.743 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:10.743 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:10.743 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:10.743 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:10.743 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:10.743 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:10.744 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:10.744 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.744 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:10.744 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.744 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.745 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:10.745 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.745 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:14.747 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:14.748 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:14.748 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:14.748 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:14.749 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:14.749 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:14.750 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:14.750 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.750 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:14.751 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.751 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.751 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:14.751 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.752 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:18.754 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:18.755 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:18.755 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:18.755 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:18.756 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:18.756 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:18.757 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:18.757 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.757 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:18.757 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.758 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.758 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:18.758 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.757 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:22.759 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:22.759 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:22.760 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:22.760 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:22.760 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:22.761 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:22.762 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:22.762 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.762 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:22.763 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.763 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.763 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:22.764 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.760 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:26.762 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:26.763 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:26.763 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:26.764 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:26.764 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:26.764 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:26.765 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:26.765 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.766 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:26.766 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.766 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.767 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:26.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.765 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:30.766 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:30.766 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:30.766 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:30.766 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.767 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.768 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:30.768 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.771 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:34.773 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:34.773 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:34.774 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:34.774 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:34.774 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:34.775 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:34.775 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:34.776 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.776 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:34.777 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.777 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.777 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:34.777 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.777 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:38.780 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:38.780 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:38.781 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:38.781 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:38.782 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:38.782 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:38.783 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:38.783 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.783 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:38.784 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.784 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.784 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:38.785 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.524 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:01:39.994 7f5f030e7b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:39.995 7f5f030e7b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19235568 disk_space_available_1k = 61854916 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:01:39.995 7f5f030e7b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:40.235 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:01:41.216 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:01:42.781 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:42.784 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:42.785 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:42.785 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:42.786 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:42.786 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:42.786 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:42.787 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:42.788 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.788 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:42.789 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.789 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.789 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:42.789 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:44.243 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19285648, partition_space_available_1k=61804836, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1288412, cpu_share=0.04, mem_res=1278412, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52372, cpu_share=0.03, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8835236, free=3550776, buffers=706712, cached=19572396, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.63, five_min_avg=3.23, fifteen_min_avg=2.18, cpu_share=4.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:01:44.243 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:44.244 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19285648 partition_space_available_1k = 61804836 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1288412 cpu_share = 0.04 mem_res = 1278412 >>contrail-database-nodemgr : << mem_virt = 52372 cpu_share = 0.03 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 8835236 free = 3550776 buffers = 706712 cached = 19572396 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.63 five_min_avg = 3.23 fifteen_min_avg = 2.18 cpu_share = 4.62 node_type = database-node >> >> 09/01/2025 02:01:44.244 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.788 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:46.790 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:46.790 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:46.790 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:46.791 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:46.791 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:46.791 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:46.792 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:46.792 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.792 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:46.792 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.793 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.793 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:46.793 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.791 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:50.794 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:50.794 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:50.795 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:50.796 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:50.796 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:50.796 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:50.797 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:50.798 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.798 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:50.799 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.799 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.799 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:50.800 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.797 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:54.798 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:54.798 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:54.799 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:54.799 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:54.799 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:54.799 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:54.800 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:54.800 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.800 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:54.800 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.800 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.800 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:54.801 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.800 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:58.802 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:58.802 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:58.802 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:58.802 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:58.802 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:58.803 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:58.803 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:58.803 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.803 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:58.804 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.804 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.804 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:01:58.804 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.805 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:02.808 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:02.808 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:02.809 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:02.809 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:02.809 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:02.810 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:02.810 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:02.811 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.811 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:02.811 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.811 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.812 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:02.812 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.811 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:06.813 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:06.814 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:06.814 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:06.815 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:06.815 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:06.815 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:06.816 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:06.816 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.817 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:06.817 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.817 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.817 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:06.818 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.818 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:10.820 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:10.820 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:10.821 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:10.821 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:10.821 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:10.822 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:10.823 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:10.823 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.823 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:10.823 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.824 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.824 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:10.824 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.822 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:14.824 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:14.825 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:14.825 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:14.826 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:14.826 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:14.826 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:14.827 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:14.828 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.828 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:14.828 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.829 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.829 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:14.829 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.828 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:18.830 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:18.831 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:18.831 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:18.832 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:18.832 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:18.832 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:18.833 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:18.833 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.834 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:18.834 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.834 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.835 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:18.835 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.833 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:22.836 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:22.836 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:22.836 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:22.837 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:22.837 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:22.837 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:22.838 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:22.839 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.839 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:22.839 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.839 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.840 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:22.840 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.836 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:26.839 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:26.839 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:26.839 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:26.840 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:26.840 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:26.840 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:26.841 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:26.841 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.842 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:26.842 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.842 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.842 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:26.843 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.842 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:30.844 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:30.844 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:30.844 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:30.845 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:30.845 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:30.845 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:30.846 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:30.846 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.846 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:30.846 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.846 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.847 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:30.847 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.848 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:34.851 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:34.851 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:34.852 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:34.852 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:34.852 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:34.853 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:34.854 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:34.854 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.854 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:34.855 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.855 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.855 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:34.855 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.852 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:38.854 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:38.855 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:38.855 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:38.856 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:38.856 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:38.856 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:38.858 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:38.858 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.859 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:38.859 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.859 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.860 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:38.860 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:39.511 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:02:40.176 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:40.177 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19422844 disk_space_available_1k = 61667640 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:02:40.177 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:40.413 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:02:41.576 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:02:42.858 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:42.863 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:42.864 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:42.865 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:42.866 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:42.866 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:42.867 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:42.871 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:42.871 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.872 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:42.872 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.872 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.873 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:42.873 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:44.647 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19424120, partition_space_available_1k=61666364, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1286936, cpu_share=0.05, mem_res=1278488, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45076, cpu_share=0.04, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8747544, free=20606484, buffers=711872, cached=2599220, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=3.12, fifteen_min_avg=2.21, cpu_share=3.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:02:44.648 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:44.649 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19424120 partition_space_available_1k = 61666364 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1286936 cpu_share = 0.05 mem_res = 1278488 >>contrail-database-nodemgr : << mem_virt = 45076 cpu_share = 0.04 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 8747544 free = 20606484 buffers = 711872 cached = 2599220 node_type = database-node >> system_cpu_usage = << one_min_avg = 3.26 five_min_avg = 3.12 fifteen_min_avg = 2.21 cpu_share = 3.5 node_type = database-node >> >> 09/01/2025 02:02:44.650 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.865 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:46.867 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:46.867 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:46.867 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:46.868 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:46.868 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:46.868 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:46.869 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:46.869 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.870 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:46.870 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.870 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.870 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:46.871 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.871 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:50.873 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:50.874 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:50.874 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:50.875 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:50.875 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:50.875 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:50.876 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:50.876 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.877 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:50.877 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.877 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.877 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:50.878 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.876 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:54.878 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:54.878 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:54.878 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:54.878 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:54.878 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:54.879 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:54.879 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:54.879 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.879 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:54.880 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.880 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.880 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:54.880 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.879 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:58.881 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:58.882 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:58.882 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:58.883 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:58.883 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:58.883 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:58.884 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:58.884 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.885 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:58.885 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.886 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.886 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:02:58.886 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.885 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:02.888 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:02.888 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:02.888 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:02.889 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:02.889 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:02.889 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:02.890 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:02.891 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.891 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:02.891 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.891 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.892 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:02.892 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.892 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:06.895 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:06.895 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:06.895 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:06.896 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:06.896 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:06.896 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:06.897 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:06.898 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.898 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:06.898 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.899 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.899 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:06.899 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.898 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:10.899 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:10.899 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:10.899 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:10.899 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:10.899 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:10.899 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:10.900 7f5f031e75e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.900 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:14.905 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:14.905 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:14.906 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:14.906 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:14.906 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:14.906 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:14.907 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:14.908 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.908 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:14.908 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.909 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.909 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:14.909 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.908 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:18.910 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:18.910 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:18.910 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:18.911 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:18.911 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:18.911 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:18.912 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:18.912 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.913 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:18.913 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.913 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.914 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:18.914 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.912 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:22.913 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:22.913 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.914 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:22.915 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.915 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.915 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:22.915 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.916 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:26.918 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:26.919 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:26.919 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:26.920 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:26.920 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:26.920 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:26.921 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:26.921 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.921 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:26.922 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.922 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.922 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:26.922 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.920 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:30.922 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:30.922 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:30.922 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:30.922 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:30.923 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:30.923 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:30.924 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:30.924 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.924 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:30.924 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.925 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.925 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:30.925 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.926 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:34.929 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:34.929 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:34.929 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:34.930 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:34.930 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:34.930 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:34.931 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:34.932 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.932 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:34.932 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.933 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.933 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:34.933 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.931 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:38.933 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:38.934 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:38.934 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:38.934 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:38.935 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:38.935 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:38.936 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:38.936 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.936 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:38.937 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.937 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.937 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:38.937 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:39.478 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:03:40.211 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:40.211 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19419952 disk_space_available_1k = 61670532 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:03:40.212 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:40.666 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:03:41.940 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:03:42.935 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:42.938 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:42.939 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:42.939 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:42.940 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:42.940 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:42.940 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:42.941 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:42.941 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.942 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:42.942 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.942 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.943 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:42.943 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:44.999 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19420080, partition_space_available_1k=61670404, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1287028, cpu_share=0.05, mem_res=1278564, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45156, cpu_share=0.04, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8613288, free=20693584, buffers=714924, cached=2643324, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.94, five_min_avg=2.8, fifteen_min_avg=2.16, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:03:45.000 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:45.001 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19420080 partition_space_available_1k = 61670404 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1287028 cpu_share = 0.05 mem_res = 1278564 >>contrail-database-nodemgr : << mem_virt = 45156 cpu_share = 0.04 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 8613288 free = 20693584 buffers = 714924 cached = 2643324 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.94 five_min_avg = 2.8 fifteen_min_avg = 2.16 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:03:45.001 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.941 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:46.942 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:46.942 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:46.942 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:46.942 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.943 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:46.944 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.946 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:50.949 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:50.949 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:50.950 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:50.950 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:50.951 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:50.951 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:50.952 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:50.953 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.953 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:50.954 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.954 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.955 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:50.955 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.953 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:54.955 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:54.955 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:54.956 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:54.956 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:54.957 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:54.957 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:54.958 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:54.958 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.958 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:54.958 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.959 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.959 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:54.959 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.958 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:58.960 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:58.961 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:58.961 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:58.962 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:58.962 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:58.962 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:58.963 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:58.963 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.963 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:58.964 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.964 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.965 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:03:58.965 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.964 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:02.966 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:02.966 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:02.966 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:02.966 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:02.966 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:02.967 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:02.967 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:02.967 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.967 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:02.967 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.968 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.968 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:02.968 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.968 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:06.970 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:06.970 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:06.970 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:06.971 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:06.971 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:06.971 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:06.972 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:06.972 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.973 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:06.973 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.973 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.973 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:06.973 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:07.752 7f5f031cda40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:10.974 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:10.977 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:10.977 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:10.978 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:10.978 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:10.978 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:10.978 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:10.979 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:10.980 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.980 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:10.980 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.981 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.981 7f5f031cda40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:10.981 7f5f031cda40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.979 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:14.986 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:14.986 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:14.987 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:14.987 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:14.987 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:14.988 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:14.988 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:14.989 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.989 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:14.990 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.990 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.990 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:14.991 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.989 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:18.991 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:18.992 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:18.992 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:18.993 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:18.993 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:18.993 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:18.994 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:18.998 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.998 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:18.998 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.999 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.999 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:18.999 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:20.668 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:22.992 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:22.994 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:22.994 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:22.994 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:22.994 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:22.994 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:22.994 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:22.995 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:22.995 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.995 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:22.995 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.995 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.995 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:22.996 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:26.996 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:26.999 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:26.999 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:26.999 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:27.000 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:27.000 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:27.000 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:27.001 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:27.001 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.002 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:27.002 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.002 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:27.002 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:27.003 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.000 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:31.002 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:31.002 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:31.003 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:31.003 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:31.003 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:31.003 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:31.004 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:31.005 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.005 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:31.005 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.005 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:31.006 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:31.006 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:33.591 7f5f030e7c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:35.004 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:35.006 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:35.006 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:35.007 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:35.007 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:35.007 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:35.008 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:35.009 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:35.009 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.009 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:35.009 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.010 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:35.010 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:35.010 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.008 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:39.010 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:39.011 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:39.011 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:39.012 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:39.012 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:39.012 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:39.013 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:39.013 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.014 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:39.014 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.014 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.014 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:39.015 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:39.415 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:04:40.212 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:40.213 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19601632 disk_space_available_1k = 61488852 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:04:40.213 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:40.667 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:04:41.937 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:04:43.012 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:43.015 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:43.015 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:43.015 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:43.016 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:43.016 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:43.016 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:43.017 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:43.018 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.018 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:43.018 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.018 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:43.019 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:43.019 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.002 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19601768, partition_space_available_1k=61488716, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1286652, cpu_share=0.05, mem_res=1278640, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45276, cpu_share=0.04, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9321840, free=19786624, buffers=720752, cached=2835904, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.47, five_min_avg=2.53, fifteen_min_avg=2.1, cpu_share=2.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:04:45.003 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:45.004 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19601768 partition_space_available_1k = 61488716 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1286652 cpu_share = 0.05 mem_res = 1278640 >>contrail-database-nodemgr : << mem_virt = 45276 cpu_share = 0.04 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 9321840 free = 19786624 buffers = 720752 cached = 2835904 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.47 five_min_avg = 2.53 fifteen_min_avg = 2.1 cpu_share = 2.12 node_type = database-node >> >> 09/01/2025 02:04:45.004 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:46.877 7f5f031e7cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:04:47.016 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:47.018 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:47.019 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:47.019 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:47.020 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:47.020 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:47.021 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:47.022 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:47.022 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.022 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:47.023 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.023 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:47.023 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:47.024 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.020 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:51.022 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:51.023 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:51.023 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:51.023 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:51.024 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:51.024 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:51.025 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:51.025 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.026 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:51.026 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.027 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:51.027 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:51.027 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.024 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:55.027 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:55.027 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:55.027 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:55.028 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:55.028 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:55.028 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:55.029 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:55.029 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.030 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:55.030 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.030 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:55.031 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:55.031 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.031 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:59.033 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:59.034 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:59.034 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:59.034 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:59.035 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:59.035 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:59.036 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:59.037 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.037 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:59.037 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.038 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.038 7f5f031e7cc0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:04:59.039 7f5f031e7cc0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:59.573 7f5f031e7cc0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:03.036 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:03.037 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:03.038 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:03.038 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:03.038 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:03.038 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:03.038 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:03.039 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:03.039 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.039 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:03.040 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.040 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:03.040 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:03.040 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.042 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:07.044 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:07.045 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:07.045 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:07.045 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:07.046 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:07.046 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:07.047 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:07.048 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.048 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:07.048 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.048 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:07.049 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:07.049 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.048 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:11.050 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:11.051 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:11.051 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:11.051 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:11.052 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:11.052 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:11.053 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:11.053 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.053 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:11.054 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.054 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:11.054 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:11.054 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:12.863 7f5f031cdb80 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:15.052 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:15.055 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:15.056 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:15.056 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:15.057 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:15.057 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:15.058 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:15.059 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:15.059 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.060 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:15.060 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.061 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:15.061 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:15.061 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.058 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:19.059 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.060 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:19.061 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.061 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:19.061 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:19.061 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.064 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:23.066 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:23.067 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:23.067 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:23.067 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:23.068 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:23.068 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:23.069 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:23.069 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.070 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:23.070 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.070 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:23.070 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:23.071 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:25.958 7f5f030e7c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:27.067 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:27.069 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:27.069 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:27.069 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:27.069 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:27.069 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:27.069 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:27.070 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:27.070 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.070 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:27.070 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.070 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:27.070 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:27.071 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.072 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:31.074 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:31.075 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:31.075 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:31.076 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:31.076 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:31.076 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:31.077 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:31.077 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.078 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:31.078 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.078 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:31.079 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:31.079 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.076 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:35.079 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:35.079 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:35.079 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:35.080 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:35.080 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:35.080 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:35.081 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:35.081 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.082 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:35.082 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.082 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:35.083 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:35.083 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:38.793 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:39.079 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:39.081 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:39.082 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:39.082 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:39.082 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:39.083 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:39.083 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:39.084 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:39.084 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.085 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:39.085 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.085 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.086 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:39.086 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:39.715 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:05:40.499 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:40.500 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19605060 disk_space_available_1k = 61485424 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:05:40.500 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:40.927 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:05:42.214 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:05:43.083 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:43.088 7f5f03d68d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:43.088 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:43.089 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:43.090 7f5f03d68d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:43.090 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:43.090 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:43.092 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:43.092 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.093 7f5f03d68d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:43.093 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.094 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:43.094 7f5f03d68d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:43.094 7f5f03d68d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.268 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19605188, partition_space_available_1k=61485296, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1287344, cpu_share=0.06, mem_res=1279020, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45364, cpu_share=0.04, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9315800, free=19787852, buffers=720968, cached=2840500, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.32, five_min_avg=2.3, fifteen_min_avg=2.05, cpu_share=1.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:05:45.269 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:45.270 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19605188 partition_space_available_1k = 61485296 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1287344 cpu_share = 0.06 mem_res = 1279020 >>contrail-database-nodemgr : << mem_virt = 45364 cpu_share = 0.04 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 9315800 free = 19787852 buffers = 720968 cached = 2840500 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.32 five_min_avg = 2.3 fifteen_min_avg = 2.05 cpu_share = 1.62 node_type = database-node >> >> 09/01/2025 02:05:45.270 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.088 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:47.090 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:47.091 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:47.091 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:47.092 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:47.092 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:47.092 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:47.093 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:47.093 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.094 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:47.094 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.094 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:47.095 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:47.095 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.092 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:51.093 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:51.094 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:51.094 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:51.094 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:51.094 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:51.094 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:51.095 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:51.643 7f5f031cd9a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:05:55.096 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:55.099 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:55.099 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:55.099 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:55.100 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:55.100 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:55.100 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:55.101 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:55.101 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.102 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:55.102 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.102 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:55.102 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:55.103 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.101 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:59.103 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:59.103 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:59.104 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:59.104 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:59.105 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:59.105 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:59.106 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:59.106 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.107 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:59.107 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.107 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:59.107 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:05:59.108 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.104 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:03.106 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:03.107 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:03.107 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:03.107 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:03.108 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:03.108 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:03.109 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:03.109 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.110 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:03.110 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.110 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:03.110 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:03.111 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:04.434 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:07.110 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:07.111 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:07.111 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:07.112 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:07.112 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:07.112 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:07.112 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:07.112 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:07.113 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.113 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:07.113 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.113 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:07.113 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:07.113 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.116 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:11.118 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:11.118 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:11.119 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:11.119 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:11.119 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:11.119 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:11.120 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:11.121 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.121 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:11.121 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.121 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:11.122 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:11.122 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.120 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:15.122 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:15.123 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:15.123 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:15.123 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:15.124 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:15.124 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:15.125 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:15.125 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.125 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:15.126 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.126 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:15.126 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:15.126 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:17.900 7f5f031cd9a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:19.124 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:19.126 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:19.127 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:19.127 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:19.127 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:19.127 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:19.128 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:19.129 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:19.129 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.129 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:19.130 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.130 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:19.130 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:19.130 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.131 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:23.132 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:23.133 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:23.133 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:23.133 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:23.134 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:23.134 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:23.135 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:23.135 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.135 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:23.136 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.136 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:23.136 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:23.136 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.136 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:27.137 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:27.137 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:27.137 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:27.137 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:27.137 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:27.137 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:27.138 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:30.748 7f5f031cd9a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:31.138 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:31.141 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:31.141 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:31.141 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:31.142 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:31.142 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:31.142 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:31.143 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:31.143 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.144 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:31.144 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.144 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:31.145 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:31.145 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.144 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:35.145 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:35.146 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:35.146 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:35.146 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:35.146 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:35.147 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:35.147 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:35.147 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.148 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:35.148 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.148 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:35.148 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:35.149 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.149 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:39.156 7f5f031e7720 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:39.157 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:39.157 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:39.157 7f5f031e7720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:39.158 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:39.158 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:39.159 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:39.159 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.159 7f5f031e7720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:39.159 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.159 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.160 7f5f031e7720 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:39.160 7f5f031e7720 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:39.604 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:06:40.438 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:40.438 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19608824 disk_space_available_1k = 61481660 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:06:40.439 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:40.837 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:06:42.296 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:06:43.158 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:43.162 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:43.162 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:43.162 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:43.163 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:43.163 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:43.163 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:43.165 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:43.165 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.165 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:43.166 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.166 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.166 7f5f031e70e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:43.167 7f5f031e70e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:43.547 7f5f031e70e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:45.355 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19609184, partition_space_available_1k=61481300, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1287608, cpu_share=0.05, mem_res=1279276, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45460, cpu_share=0.04, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9337888, free=19761308, buffers=721012, cached=2844912, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.39, five_min_avg=2.16, fifteen_min_avg=2.02, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:06:45.356 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:45.357 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19609184 partition_space_available_1k = 61481300 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1287608 cpu_share = 0.05 mem_res = 1279276 >>contrail-database-nodemgr : << mem_virt = 45460 cpu_share = 0.04 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 9337888 free = 19761308 buffers = 721012 cached = 2844912 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.39 five_min_avg = 2.16 fifteen_min_avg = 2.02 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:06:45.357 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.163 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:47.165 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:47.166 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:47.166 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:47.166 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:47.166 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:47.167 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:47.167 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:47.168 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.169 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:47.169 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.169 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:47.170 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:47.170 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.169 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:51.171 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:51.171 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:51.171 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:51.172 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:51.172 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:51.173 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:51.174 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:51.174 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.175 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:51.175 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.175 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:51.176 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:51.176 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.172 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:55.174 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:55.175 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:55.175 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:55.176 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:55.176 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:55.176 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:55.177 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:55.178 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.178 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:55.178 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.179 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:55.179 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:55.180 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:56.442 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:06:59.175 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:59.178 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:59.179 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:59.179 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:59.179 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:59.180 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:59.180 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:59.181 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:59.181 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.181 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:59.182 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.182 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:59.182 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:06:59.183 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.180 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:03.183 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:03.184 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:03.185 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:03.185 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:03.186 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:03.187 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:03.188 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:03.189 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.189 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:03.190 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.190 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:03.191 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:03.191 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.188 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:07.190 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:07.191 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:07.191 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:07.191 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:07.192 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:07.192 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:07.193 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:07.193 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.194 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:07.194 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.194 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:07.195 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:07.195 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:09.275 7f5f030e7c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:11.193 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:11.195 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:11.196 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:11.196 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:11.196 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:11.197 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:11.197 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:11.198 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:11.198 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.198 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:11.198 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.199 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:11.199 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:11.199 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.199 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:15.200 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:15.200 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:15.200 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:15.200 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:15.201 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:15.201 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:15.201 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:15.201 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.201 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:15.202 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.202 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:15.202 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:15.202 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.202 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:19.204 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:19.205 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:19.205 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:19.205 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:19.206 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:19.206 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:19.207 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:19.207 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.208 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:19.208 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.208 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:19.209 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:19.209 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.602 7f5f031e77c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:23.205 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:23.207 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:23.207 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:23.207 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:23.208 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:23.208 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:23.208 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:23.209 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:23.209 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.209 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:23.209 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.210 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:23.210 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:23.210 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.212 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:27.214 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:27.214 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:27.215 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:27.215 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:27.216 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:27.216 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:27.217 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:27.217 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.218 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:27.218 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.218 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:27.219 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:27.219 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.216 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:31.218 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:31.218 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:31.218 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:31.219 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.220 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:31.220 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:31.220 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.220 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:35.222 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:35.222 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:35.222 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:35.223 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:35.223 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:35.224 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:35.225 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:35.225 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.225 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:35.226 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.226 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.226 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:35.227 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:35.614 7f5f031cdea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:39.224 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:39.234 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:39.235 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:39.235 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:39.236 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:39.236 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:39.236 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:39.237 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:39.237 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.238 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:39.238 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.238 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.239 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:39.239 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:39.620 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:07:40.446 7f5f031cd400 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:40.447 7f5f031cd400 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19612076 disk_space_available_1k = 61478408 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:07:40.447 7f5f031cd400 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:40.906 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:07:42.106 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:07:43.235 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:43.238 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:43.239 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:43.239 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:43.240 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:43.240 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:43.240 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:43.241 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:43.241 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.242 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:43.242 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.242 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:43.243 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:43.243 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.173 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19612216, partition_space_available_1k=61478268, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1287816, cpu_share=0.05, mem_res=1279428, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45552, cpu_share=0.04, mem_res=42776, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9334504, free=19761332, buffers=721052, cached=2848232, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.72, five_min_avg=2.16, fifteen_min_avg=2.03, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:07:45.174 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:45.174 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19612216 partition_space_available_1k = 61478268 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1287816 cpu_share = 0.05 mem_res = 1279428 >>contrail-database-nodemgr : << mem_virt = 45552 cpu_share = 0.04 mem_res = 42776 >> } system_mem_usage = << total = 32665120 used = 9334504 free = 19761332 buffers = 721052 cached = 2848232 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.72 five_min_avg = 2.16 fifteen_min_avg = 2.03 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:07:45.175 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.241 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:47.244 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:47.244 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:47.244 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:47.245 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:47.245 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:47.245 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:47.246 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:47.246 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.247 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:47.247 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.247 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:47.247 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:47.248 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:48.622 7f5f031e7a40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:07:51.244 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:51.247 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:51.247 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:51.248 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:51.248 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:51.249 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:51.249 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:51.250 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:51.251 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.251 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:51.251 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.251 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:51.252 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:51.252 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.248 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:55.251 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:55.251 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:55.252 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:55.252 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:55.253 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:55.253 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:55.254 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:55.255 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.256 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:55.256 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.256 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:55.257 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:55.257 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.252 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:59.254 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:59.254 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:59.254 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:59.255 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:59.255 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:59.255 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:59.255 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:59.256 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.256 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:59.256 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.256 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:59.257 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:07:59.257 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:01.516 7f5f031cd9a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:03.257 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:03.259 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:03.260 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:03.260 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:03.261 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:03.261 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:03.261 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:03.262 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:03.263 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.263 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:03.263 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.264 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:03.264 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:03.264 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.262 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:07.265 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:07.265 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:07.266 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:07.266 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:07.266 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:07.267 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:07.268 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:07.268 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.269 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:07.269 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.269 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:07.270 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:07.270 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.268 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:11.270 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:11.271 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:11.271 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:11.271 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:11.272 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:11.272 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:11.273 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:11.273 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.273 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:11.274 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.274 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:11.274 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:11.274 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:14.427 7f5f031e7860 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:15.272 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:15.274 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:15.275 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:15.275 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:15.275 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:15.276 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:15.276 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:15.277 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:15.277 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:15.278 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:15.278 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:15.278 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:15.279 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:15.279 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.275 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:19.277 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:19.277 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:19.278 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:19.278 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:19.278 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:19.279 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:19.279 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:19.280 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.280 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:19.281 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.281 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:19.281 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:19.281 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.280 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:23.283 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:23.283 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:23.283 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:23.284 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:23.285 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:23.285 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:23.286 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:23.286 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.286 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:23.286 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.287 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:23.287 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:23.287 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.284 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:27.287 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:27.288 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:27.288 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:27.288 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:27.289 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:27.289 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:27.290 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:27.290 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.290 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:27.291 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.291 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.291 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:27.291 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:27.851 7f5f033af0e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:31.292 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:31.294 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:31.295 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:31.295 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:31.296 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:31.296 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:31.296 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:31.297 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:31.298 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:31.298 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:31.298 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:31.299 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:31.299 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:31.299 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.295 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:35.297 7f5f031e79a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:35.298 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:35.298 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:35.298 7f5f031e79a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:35.299 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:35.299 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:35.300 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:35.300 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.300 7f5f031e79a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:35.300 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.301 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:35.301 7f5f031e79a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:35.301 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.300 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:39.309 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:39.310 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:39.310 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:39.311 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:39.311 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:39.311 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:39.312 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:39.313 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.313 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:39.313 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.314 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.314 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:39.314 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:39.579 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:08:40.261 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:40.261 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19617604 disk_space_available_1k = 61472880 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:08:40.261 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:40.568 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:08:40.697 7f5f031cd540 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:41.809 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:08:43.311 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:43.315 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:43.315 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:43.315 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:43.316 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:43.316 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:43.317 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:43.318 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:43.318 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.319 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:43.319 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.319 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:43.320 7f5f031e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:43.320 7f5f031e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:44.874 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19616060, partition_space_available_1k=61474424, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1287924, cpu_share=0.05, mem_res=1279504, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45656, cpu_share=0.04, mem_res=42784, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9381760, free=19705508, buffers=721216, cached=2856636, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.4, five_min_avg=1.99, fifteen_min_avg=1.98, cpu_share=1.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:08:44.875 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:44.876 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19616060 partition_space_available_1k = 61474424 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1287924 cpu_share = 0.05 mem_res = 1279504 >>contrail-database-nodemgr : << mem_virt = 45656 cpu_share = 0.04 mem_res = 42784 >> } system_mem_usage = << total = 32665120 used = 9381760 free = 19705508 buffers = 721216 cached = 2856636 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.4 five_min_avg = 1.99 fifteen_min_avg = 1.98 cpu_share = 1.88 node_type = database-node >> >> 09/01/2025 02:08:44.876 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.316 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:47.318 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:47.319 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:47.319 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:47.320 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:47.320 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:47.321 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:47.321 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:47.322 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.322 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:47.322 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.322 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:47.323 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:47.323 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.320 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:51.322 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:51.323 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:51.323 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:51.324 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:51.324 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:51.325 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:51.326 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:51.326 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.327 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:51.327 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.327 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:51.328 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:51.328 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:53.910 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:08:55.325 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:55.327 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:55.327 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:55.328 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:55.328 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:55.328 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:55.329 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:55.330 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:55.330 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.330 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:55.330 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.331 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:55.331 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:55.331 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.331 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:59.332 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:59.332 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:59.332 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.333 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:59.334 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:08:59.334 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.337 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:03.339 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:03.339 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:03.339 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:03.340 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:03.340 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:03.340 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:03.341 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:03.341 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.341 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:03.342 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.342 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:03.342 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:03.342 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.078 7f5f033af0e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:07.339 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:07.341 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:07.341 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:07.341 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:07.342 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:07.342 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:07.342 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:07.342 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:07.343 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.343 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:07.343 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.343 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:07.343 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:07.344 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.344 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:11.347 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:11.347 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:11.347 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:11.348 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:11.348 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:11.348 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:11.349 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:11.350 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.350 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:11.350 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.351 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:11.351 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:11.351 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.347 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:15.350 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:15.351 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:15.351 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:15.352 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:15.352 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:15.352 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:15.354 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:15.354 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.355 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:15.355 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.355 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:15.356 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:15.356 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.352 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:19.355 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:19.356 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:19.356 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:19.357 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:19.357 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:19.358 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:19.359 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:19.359 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.360 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:19.360 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.361 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:19.361 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:19.361 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:20.163 7f5f033af0e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:23.358 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:23.361 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:23.361 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:23.361 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:23.362 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:23.362 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:23.362 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:23.363 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:23.363 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:23.364 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:23.364 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:23.364 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:23.365 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:23.365 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.365 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:27.368 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:27.368 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:27.368 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:27.369 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:27.369 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:27.369 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:27.370 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:27.370 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.371 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:27.371 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.371 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:27.371 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:27.372 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.368 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:31.371 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:31.371 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:31.371 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:31.372 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:31.372 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:31.372 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:31.373 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:31.373 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.374 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:31.374 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.374 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:31.374 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:31.375 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:33.419 7f5f031e7860 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:35.373 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:35.375 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:35.375 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:35.375 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:35.375 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:35.375 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:35.375 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:35.376 7f5f031e7860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.375 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:39.377 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:39.377 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:39.377 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:39.377 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:39.377 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:39.377 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:39.378 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:39.378 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.378 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:39.379 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.379 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.379 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:39.379 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:39.513 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:09:40.351 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:40.352 7f5f033af360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19619260 disk_space_available_1k = 61471224 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:09:40.353 7f5f033af360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:40.715 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:09:42.033 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:09:43.378 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:43.382 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:43.382 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:43.383 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:43.383 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:43.383 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:43.384 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:43.385 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:43.385 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:43.386 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:43.386 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:43.386 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:43.387 7f5f031e72c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:43.387 7f5f031e72c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.121 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19619404, partition_space_available_1k=61471080, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1287736, cpu_share=0.05, mem_res=1279352, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45748, cpu_share=0.04, mem_res=42784, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9381676, free=19706156, buffers=721296, cached=2855992, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.52, five_min_avg=1.89, fifteen_min_avg=1.94, cpu_share=1.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:09:45.122 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:45.123 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19619404 partition_space_available_1k = 61471080 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1287736 cpu_share = 0.05 mem_res = 1279352 >>contrail-database-nodemgr : << mem_virt = 45748 cpu_share = 0.04 mem_res = 42784 >> } system_mem_usage = << total = 32665120 used = 9381676 free = 19706156 buffers = 721296 cached = 2855992 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.52 five_min_avg = 1.89 fifteen_min_avg = 1.94 cpu_share = 1.62 node_type = database-node >> >> 09/01/2025 02:09:45.123 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:46.347 7f5f031cd220 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:47.384 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:47.386 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:47.387 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:47.387 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:47.388 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:47.388 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:47.389 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:47.389 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:47.390 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.390 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:47.391 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.391 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:47.391 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:47.392 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.391 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:51.400 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:51.400 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:51.400 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:51.401 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:51.401 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:51.401 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:51.402 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:51.403 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.403 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:51.403 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.403 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:51.404 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:51.404 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.404 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:55.405 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:55.406 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:55.406 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:55.406 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:55.406 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:55.406 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:55.406 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:55.407 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.407 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:55.407 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.407 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:55.407 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:55.407 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.354 7f5f031e77c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:09:59.406 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:59.408 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:59.409 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:59.409 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:59.409 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:59.410 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:59.410 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:59.411 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:59.411 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.411 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:59.412 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.412 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:59.412 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:09:59.412 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.411 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:03.414 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:03.414 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:03.415 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:03.415 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:03.415 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:03.416 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:03.416 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:03.417 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.417 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:03.417 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.418 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:03.418 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:03.418 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.416 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:07.418 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:07.418 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:07.418 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:07.419 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:07.419 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:07.419 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:07.420 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:07.421 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.421 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:07.421 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:07.422 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:07.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.420 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:11.421 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:11.421 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:11.421 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:11.421 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:11.421 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:11.421 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:11.422 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:12.313 7f5f031cd5e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:15.425 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:15.426 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:15.427 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:15.427 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:15.427 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:15.427 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:15.428 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:15.428 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:15.428 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:15.429 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:15.429 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:15.429 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:15.429 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:15.429 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.431 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:19.434 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:19.435 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:19.435 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:19.435 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:19.436 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:19.436 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:19.437 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:19.437 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.438 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:19.438 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.438 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:19.438 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:19.439 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.435 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:23.438 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:23.438 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:23.438 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:23.439 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:23.439 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:23.439 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:23.440 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:23.441 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.441 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:23.441 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.442 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:23.442 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:23.442 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:25.186 7f5f031cdae0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:27.440 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:27.443 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:27.443 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:27.443 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:27.444 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:27.444 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:27.445 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:27.445 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:27.446 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.446 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:27.446 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.447 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:27.447 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:27.447 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.445 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:31.446 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:31.446 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:31.446 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:31.446 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:31.447 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:31.447 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:31.447 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:31.447 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.447 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:31.448 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.448 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:31.448 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:31.448 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.451 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:35.453 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:35.453 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:35.454 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:35.454 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:35.454 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:35.454 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:35.455 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:35.455 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.456 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:35.456 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.456 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:35.457 7f5f031cdae0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:35.457 7f5f031cdae0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.533 7f5f033af360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:39.454 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:39.456 7f5f033af360 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:39.457 7f5f033af360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:39.457 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:39.457 7f5f033af360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:39.458 7f5f033af360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:39.458 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:39.459 7f5f033af360 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:39.459 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.459 7f5f033af360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:39.459 7f5f033af360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.460 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.460 7f5f033af360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:39.460 7f5f033af360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:39.594 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:10:40.407 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:40.408 7f5f033af360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19622792 disk_space_available_1k = 61467692 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:10:40.408 7f5f033af360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:40.853 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:10:42.107 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:10:43.457 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:43.459 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:43.459 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:43.460 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:43.460 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:43.460 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:43.460 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:43.460 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:43.460 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:43.461 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:43.461 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:43.461 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:43.461 7f5f031cde00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:43.461 7f5f031cde00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:45.176 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19622944, partition_space_available_1k=61467540, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1288304, cpu_share=0.05, mem_res=1279644, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45840, cpu_share=0.04, mem_res=42788, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9377772, free=19706948, buffers=721340, cached=2859060, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.11, five_min_avg=1.72, fifteen_min_avg=1.88, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:10:45.177 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:45.178 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19622944 partition_space_available_1k = 61467540 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1288304 cpu_share = 0.05 mem_res = 1279644 >>contrail-database-nodemgr : << mem_virt = 45840 cpu_share = 0.04 mem_res = 42788 >> } system_mem_usage = << total = 32665120 used = 9377772 free = 19706948 buffers = 721340 cached = 2859060 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.11 five_min_avg = 1.72 fifteen_min_avg = 1.88 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:10:45.179 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.460 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:47.463 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:47.463 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:47.464 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:47.464 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:47.464 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:47.465 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:47.465 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:47.466 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.466 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:47.466 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.467 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:47.467 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:47.467 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.420 7f5f031e7180 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:10:51.464 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:51.466 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:51.466 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:51.466 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:51.467 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:51.467 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:51.467 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:51.468 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:51.468 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.468 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:51.468 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.469 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:51.469 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:51.469 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.466 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:55.469 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:55.469 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:55.469 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:55.470 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:55.470 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:55.470 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:55.471 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:55.472 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.472 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:55.472 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.473 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:55.473 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:55.473 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.472 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:59.475 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:59.475 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:59.475 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:59.476 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:59.476 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:59.477 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:59.477 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:59.478 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.478 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:59.479 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.479 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:59.479 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:10:59.480 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.477 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:03.479 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:03.479 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:03.479 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:03.480 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:03.480 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:03.480 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:03.481 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:03.482 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.482 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:03.482 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.482 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:03.483 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:03.483 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:04.148 7f5f031e7540 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:07.481 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:07.483 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:07.483 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:07.484 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:07.484 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:07.484 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:07.485 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:07.485 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:07.486 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:07.486 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:07.486 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:07.486 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:07.487 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:07.487 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.485 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:11.486 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:11.486 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:11.487 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:11.487 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:11.487 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:11.487 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:11.487 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:11.487 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.488 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:11.488 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.488 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:11.488 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:11.488 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.489 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:15.493 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:15.493 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:15.493 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:15.494 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:15.494 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:15.494 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:15.495 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:15.496 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.496 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:15.496 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.496 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:15.497 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:15.497 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:17.060 7f5f031e7e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:19.496 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:19.498 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:19.499 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:19.499 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:19.500 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:19.500 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:19.500 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:19.501 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:19.501 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:19.502 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:19.502 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:19.502 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:19.502 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:19.503 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.500 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:23.505 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:23.505 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:23.505 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:23.506 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:23.506 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:23.506 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:23.507 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:23.507 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.507 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:23.507 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.507 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:23.507 7f5f031cd540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:23.508 7f5f031cd540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.510 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:27.512 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:27.512 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:27.513 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:27.513 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:27.513 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:27.513 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:27.514 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:27.514 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.515 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:27.515 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.515 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:27.516 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:27.516 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:29.911 7f5f030e7c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:31.513 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:31.515 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:31.516 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:31.516 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:31.517 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:31.517 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:31.517 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:31.518 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:31.519 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:31.519 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:31.519 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:31.519 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:31.520 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:31.520 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.516 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:35.519 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:35.519 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:35.520 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:35.520 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:35.520 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:35.521 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:35.521 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:35.522 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.522 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:35.522 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.523 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:35.523 7f5f030e7c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:35.523 7f5f030e7c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.502 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:11:39.519 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:39.520 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:39.521 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:39.521 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:39.521 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:39.521 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:39.521 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:39.522 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:39.522 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.522 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:39.522 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.522 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:39.523 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:39.523 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:40.214 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:40.214 7f5f031e79a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19626612 disk_space_available_1k = 61463872 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:11:40.215 7f5f031e79a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:40.583 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:11:41.914 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:11:43.137 7f5f031e7a40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:43.522 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:43.525 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:43.526 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:43.526 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:43.527 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:43.527 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:43.527 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:43.528 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:43.529 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.529 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:43.529 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.530 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:43.530 7f5f031e7a40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:43.530 7f5f031e7a40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:44.961 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19626948, partition_space_available_1k=61463536, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1288424, cpu_share=0.05, mem_res=1279732, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45932, cpu_share=0.04, mem_res=42788, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9394976, free=19685340, buffers=721388, cached=2863416, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.07, five_min_avg=1.59, fifteen_min_avg=1.82, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:11:44.961 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:44.962 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19626948 partition_space_available_1k = 61463536 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1288424 cpu_share = 0.05 mem_res = 1279732 >>contrail-database-nodemgr : << mem_virt = 45932 cpu_share = 0.04 mem_res = 42788 >> } system_mem_usage = << total = 32665120 used = 9394976 free = 19685340 buffers = 721388 cached = 2863416 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.07 five_min_avg = 1.59 fifteen_min_avg = 1.82 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:11:44.962 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.528 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:47.529 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:47.530 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:47.530 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:47.530 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:47.530 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:47.530 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:47.531 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:47.531 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.531 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:47.531 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.531 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:47.531 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:47.532 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.532 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:51.535 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:51.535 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:51.535 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:51.536 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:51.536 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:51.536 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:51.537 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:51.537 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.537 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:51.538 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.538 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:51.538 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:51.538 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.536 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:55.539 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:55.539 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:55.540 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:55.540 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:55.540 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:55.541 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:55.542 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:55.542 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.543 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:55.543 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.543 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:55.544 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:55.544 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:56.167 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:11:59.540 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:59.542 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:59.543 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:59.543 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:59.543 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:59.544 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:59.544 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:59.545 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:59.545 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:59.545 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:59.546 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:59.546 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:59.546 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:11:59.547 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.546 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:03.548 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:03.548 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:03.549 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:03.549 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:03.549 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:03.549 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:03.550 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:03.550 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.550 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:03.550 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.551 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:03.551 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:03.551 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.553 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:07.573 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:07.574 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:07.574 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:07.574 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:07.575 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:07.575 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:07.576 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:07.576 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.577 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:07.577 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.577 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:07.578 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:07.578 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:09.414 7f5f031e7e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:11.575 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:11.577 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:11.577 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:11.577 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:11.577 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:11.578 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:11.578 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:11.578 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:11.579 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:11.579 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:11.579 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:11.579 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:11.580 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:11.580 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.580 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:15.583 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:15.584 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:15.584 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:15.585 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:15.585 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:15.585 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:15.586 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:15.587 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.587 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:15.587 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.588 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:15.588 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:15.588 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.584 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:19.587 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:19.587 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:19.587 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:19.588 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:19.588 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:19.589 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:19.590 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:19.590 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.590 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:19.591 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.591 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:19.591 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:19.592 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.473 7f5f031cdea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:23.589 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:23.591 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:23.592 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:23.593 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:23.593 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:23.593 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:23.594 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:23.595 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:23.595 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:23.595 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:23.596 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:23.596 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:23.596 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:23.597 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.595 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:27.598 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:27.598 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:27.598 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:27.599 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:27.599 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:27.600 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:27.601 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:27.601 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.601 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:27.602 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.602 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:27.603 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:27.603 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.602 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:31.604 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:31.605 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:31.605 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:31.606 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:31.606 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:31.607 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:31.608 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:31.608 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.609 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:31.609 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.609 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:31.610 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:31.610 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.470 7f5f031e7180 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:35.606 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:35.608 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:35.608 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:35.609 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:35.609 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:35.609 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:35.610 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:35.610 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:35.611 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.611 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:35.611 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.612 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:35.612 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:35.612 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.590 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:12:39.609 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:39.610 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:39.611 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:39.611 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:39.612 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:39.612 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:39.612 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:39.613 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:39.613 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.614 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:39.614 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.614 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:39.614 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:39.615 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:40.379 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:40.380 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19629996 disk_space_available_1k = 61460488 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:12:40.380 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:40.828 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:12:42.010 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:12:43.612 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:43.616 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:43.616 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:43.616 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:43.617 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:43.617 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:43.617 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:43.618 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:43.619 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:43.619 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:43.619 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:43.620 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:43.620 7f5f031e74a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:43.620 7f5f031e74a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:45.058 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19630132, partition_space_available_1k=61460352, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1288432, cpu_share=0.05, mem_res=1279732, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46416, cpu_share=0.04, mem_res=43176, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9400504, free=19670844, buffers=721448, cached=2872324, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.83, five_min_avg=1.43, fifteen_min_avg=1.75, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:12:45.059 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:45.059 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19630132 partition_space_available_1k = 61460352 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1288432 cpu_share = 0.05 mem_res = 1279732 >>contrail-database-nodemgr : << mem_virt = 46416 cpu_share = 0.04 mem_res = 43176 >> } system_mem_usage = << total = 32665120 used = 9400504 free = 19670844 buffers = 721448 cached = 2872324 node_type = database-node >> system_cpu_usage = << one_min_avg = 0.83 five_min_avg = 1.43 fifteen_min_avg = 1.75 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:12:45.059 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.618 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:47.621 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:47.622 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:47.622 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:47.622 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:47.623 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:47.623 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:47.624 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:47.624 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.624 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:47.625 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.625 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:47.625 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:47.625 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:48.900 7f5f033af0e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:12:51.624 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:51.627 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:51.627 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:51.628 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:51.628 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:51.629 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:51.629 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:51.630 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:51.630 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:51.631 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:51.631 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:51.631 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:51.632 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:51.632 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.628 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:55.630 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:55.631 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:55.631 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:55.631 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:55.632 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:55.632 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:55.633 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:55.634 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.634 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:55.635 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.635 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:55.635 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:55.636 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.632 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:59.634 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:59.635 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:59.635 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:59.636 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:59.636 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:59.637 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:59.638 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:59.638 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.639 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:59.639 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.640 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:59.640 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:12:59.641 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:01.776 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:03.637 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:03.640 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:03.640 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:03.640 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:03.641 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:03.641 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:03.641 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:03.642 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:03.642 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:03.643 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:03.643 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:03.643 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:03.643 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:03.644 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.640 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:07.643 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:07.643 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:07.643 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:07.643 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:07.643 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:07.643 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:07.644 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.645 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:11.647 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:11.648 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:11.648 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:11.649 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:11.649 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:11.649 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:11.650 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:11.650 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.651 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:11.651 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.651 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:11.651 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:11.652 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:14.858 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:15.649 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:15.651 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:15.652 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:15.652 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:15.653 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:15.653 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:15.653 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:15.654 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:15.654 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:15.655 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:15.655 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:15.655 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:15.655 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:15.656 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.656 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:19.659 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:19.659 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:19.659 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:19.660 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:19.660 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:19.660 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:19.661 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:19.661 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.662 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:19.662 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.662 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:19.663 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:19.663 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.663 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:23.664 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:23.664 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:23.665 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.666 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:27.669 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:27.669 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:27.669 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:27.670 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:27.670 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:27.670 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:27.671 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:27.671 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.672 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:27.672 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.672 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.672 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:27.673 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:27.847 7f5f031e7180 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:31.673 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:31.675 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:31.676 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:31.676 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:31.677 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:31.677 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:31.677 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:31.678 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:31.678 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:31.679 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:31.679 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:31.679 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:31.680 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:31.680 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.681 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:35.683 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:35.684 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:35.684 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:35.684 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:35.685 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:35.685 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:35.686 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:35.686 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.687 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:35.687 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.687 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:35.687 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:35.688 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.500 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:13:39.686 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:39.693 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:39.693 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:39.694 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:39.694 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:39.694 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:39.695 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:39.696 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:39.696 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.696 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:39.697 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.697 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:39.697 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:39.697 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:40.268 7f5f033af360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:40.269 7f5f033af360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19634372 disk_space_available_1k = 61456112 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:13:40.269 7f5f033af360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:40.643 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:13:40.986 7f5f033af360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:41.820 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:13:43.695 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:43.697 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:43.697 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:43.697 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:43.698 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:43.698 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:43.698 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:43.699 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:43.699 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:43.699 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:43.699 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:43.699 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:43.700 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:43.700 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:44.874 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19633376, partition_space_available_1k=61457108, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1288636, cpu_share=0.05, mem_res=1279884, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46504, cpu_share=0.04, mem_res=43176, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9390116, free=19683184, buffers=721492, cached=2870328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.27, five_min_avg=1.47, fifteen_min_avg=1.75, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:13:44.875 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:44.876 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19633376 partition_space_available_1k = 61457108 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1288636 cpu_share = 0.05 mem_res = 1279884 >>contrail-database-nodemgr : << mem_virt = 46504 cpu_share = 0.04 mem_res = 43176 >> } system_mem_usage = << total = 32665120 used = 9390116 free = 19683184 buffers = 721492 cached = 2870328 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.27 five_min_avg = 1.47 fifteen_min_avg = 1.75 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:13:44.877 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.700 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:47.702 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:47.703 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:47.703 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:47.704 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:47.704 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:47.704 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:47.705 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:47.705 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.706 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:47.706 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.706 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:47.706 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:47.707 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.704 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:51.707 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:51.708 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:51.708 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:51.709 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:51.709 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:51.710 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:51.711 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:51.711 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.712 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:51.712 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.713 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:51.713 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:51.714 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:54.302 7f5f031cd220 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:13:55.708 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:55.711 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:55.711 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:55.711 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:55.712 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:55.712 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:55.712 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:55.713 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:55.714 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:55.714 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:55.714 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:55.714 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:55.715 7f5f031cd220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:55.715 7f5f031cd220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.713 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:13:59.715 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:13:59.715 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:13:59.715 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:59.716 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:13:59.716 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:13:59.716 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:13:59.717 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:13:59.717 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.717 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:13:59.717 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.717 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:13:59.717 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:13:59.718 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.719 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:03.721 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:03.721 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:03.722 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:03.722 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:03.722 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:03.723 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:03.723 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:03.724 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.724 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:03.724 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.725 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:03.725 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:03.725 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.139 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:07.723 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:07.726 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:07.726 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:07.726 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:07.727 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:07.727 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:07.728 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:07.729 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:07.729 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.730 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:07.730 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.730 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:07.731 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:07.731 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.728 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:11.729 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:11.730 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:11.730 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:11.730 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:11.730 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:11.730 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:11.731 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.732 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:15.734 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:15.735 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:15.735 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:15.736 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:15.736 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:15.736 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:15.737 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:15.738 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.738 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:15.738 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.739 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:15.739 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:15.739 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.736 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:19.738 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:19.739 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:19.739 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:19.739 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:19.739 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:19.739 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:19.740 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:19.740 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.740 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:19.740 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.740 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:19.740 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:19.741 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:20.245 7f5f030e7e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:23.743 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:23.745 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:23.745 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:23.745 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:23.745 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:23.746 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:23.746 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:23.746 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:23.747 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:23.747 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:23.747 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:23.747 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:23.748 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:23.748 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.746 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:27.748 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:27.748 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:27.748 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:27.748 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:27.748 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:27.749 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:27.749 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:27.749 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.749 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:27.749 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.749 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:27.750 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:27.750 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.752 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:31.754 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:31.754 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:31.754 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:31.755 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:31.755 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:31.755 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:31.755 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:31.755 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.756 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:31.756 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.756 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:31.756 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:31.756 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:33.306 7f5f031cd9a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:35.757 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:35.760 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:35.760 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:35.760 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:35.761 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:35.761 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:35.761 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:35.762 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:35.762 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:35.763 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:35.763 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:35.763 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:35.764 7f5f031cd9a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:35.764 7f5f031cd9a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.535 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:14:39.762 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:39.768 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:39.768 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:39.769 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:39.769 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:39.769 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:39.769 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:39.770 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:39.770 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.771 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:39.771 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.771 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:39.771 7f5f031e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:39.771 7f5f031e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:40.300 7f5f031cd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:40.301 7f5f031cd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19636700 disk_space_available_1k = 61453784 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:14:40.301 7f5f031cd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:40.732 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:14:41.990 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:14:43.769 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:43.772 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:43.773 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:43.773 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:43.774 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:43.774 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:43.774 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:43.775 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:43.776 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:43.776 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:43.777 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:43.777 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:43.777 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:43.778 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:45.074 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19636756, partition_space_available_1k=61453728, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1288748, cpu_share=0.05, mem_res=1279964, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46596, cpu_share=0.04, mem_res=43176, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9404180, free=19662296, buffers=721768, cached=2876876, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.53, five_min_avg=1.55, fifteen_min_avg=1.76, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:14:45.075 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:45.076 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19636756 partition_space_available_1k = 61453728 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1288748 cpu_share = 0.05 mem_res = 1279964 >>contrail-database-nodemgr : << mem_virt = 46596 cpu_share = 0.04 mem_res = 43176 >> } system_mem_usage = << total = 32665120 used = 9404180 free = 19662296 buffers = 721768 cached = 2876876 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.53 five_min_avg = 1.55 fifteen_min_avg = 1.76 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:14:45.076 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:46.145 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:47.775 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:47.777 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:47.777 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:47.778 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:47.778 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:47.778 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:47.779 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:47.779 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:47.780 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:47.780 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:47.780 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:47.781 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:47.781 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:47.781 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:51.781 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:51.783 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:51.784 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:51.784 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:51.785 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:51.785 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:51.785 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:51.786 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:51.787 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:51.787 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:51.787 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:51.788 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:51.788 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:51.789 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:55.788 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:55.791 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:55.791 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:55.791 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:55.792 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:55.792 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:55.793 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:55.794 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:55.794 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:55.794 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:55.795 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:55.795 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:55.795 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:55.796 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:59.787 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:14:59.791 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:14:59.792 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:14:59.792 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:14:59.792 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:59.792 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:14:59.793 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:14:59.793 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:14:59.793 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:14:59.793 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:59.793 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:14:59.793 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:59.794 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:14:59.794 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:14:59.794 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:03.796 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:03.798 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:03.798 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:03.798 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:03.798 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:03.799 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:03.799 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:03.799 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:03.799 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:03.800 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:03.800 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:03.800 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:03.800 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:03.800 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:07.800 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:07.802 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:07.803 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:07.803 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:07.803 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:07.803 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:07.804 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:07.804 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:07.805 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:07.805 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:07.805 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:07.805 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:07.806 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:07.806 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:11.806 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:11.807 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:11.807 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:11.808 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:11.808 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:11.808 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:11.808 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:11.808 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:11.809 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:11.809 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:11.809 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:11.809 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:11.809 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:11.809 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:12.864 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:15.811 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:15.813 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:15.813 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:15.814 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:15.814 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:15.814 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:15.814 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:15.815 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:15.815 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:15.816 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:15.816 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:15.816 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:15.817 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:15.817 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:19.817 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:19.820 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:19.820 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:19.821 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:19.821 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:19.821 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:19.821 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:19.822 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:19.823 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:19.823 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:19.823 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:19.824 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:19.824 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:19.824 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:23.824 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:23.826 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:23.827 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:23.827 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:23.828 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:23.828 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:23.828 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:23.829 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:23.829 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:23.829 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:23.830 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:23.830 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:23.830 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:23.830 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:25.749 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:27.829 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:27.831 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:27.832 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:27.832 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:27.832 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:27.832 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:27.833 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:27.833 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:27.834 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:27.834 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:27.834 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:27.834 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:27.835 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:27.835 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:31.835 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:31.838 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:31.838 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:31.839 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:31.839 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:31.840 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:31.840 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:31.842 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:31.842 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:31.843 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:31.843 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:31.843 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:31.844 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:31.844 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:35.840 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:35.843 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:35.843 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:35.843 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:35.844 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:35.844 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:35.844 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:35.845 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:35.845 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:35.846 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:35.846 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:35.846 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:35.846 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:35.847 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:38.663 7f5f030e7ea0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:39.592 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:15:39.843 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:39.848 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:39.849 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:39.849 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:39.849 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:39.849 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:39.850 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:39.850 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:39.851 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:39.851 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:39.851 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:39.851 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:39.851 7f5f031e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:39.852 7f5f031e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:40.373 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:40.374 7f5f031cdb80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19640228 disk_space_available_1k = 61450256 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:15:40.374 7f5f031cdb80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:40.795 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:15:42.062 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:15:43.849 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:43.851 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:43.851 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:43.851 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:43.852 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:43.852 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:43.852 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:43.852 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:43.852 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:43.853 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:43.853 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:43.853 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:43.853 7f5f031e7540 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:43.853 7f5f031e7540 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:45.113 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19640276, partition_space_available_1k=61450208, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1292968, cpu_share=0.06, mem_res=1283992, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46720, cpu_share=0.04, mem_res=43176, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9373344, free=19689616, buffers=721852, cached=2880308, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.71, five_min_avg=1.59, fifteen_min_avg=1.76, cpu_share=1.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:15:45.114 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:45.115 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19640276 partition_space_available_1k = 61450208 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1292968 cpu_share = 0.06 mem_res = 1283992 >>contrail-database-nodemgr : << mem_virt = 46720 cpu_share = 0.04 mem_res = 43176 >> } system_mem_usage = << total = 32665120 used = 9373344 free = 19689616 buffers = 721852 cached = 2880308 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.71 five_min_avg = 1.59 fifteen_min_avg = 1.76 cpu_share = 1.88 node_type = database-node >> >> 09/01/2025 02:15:45.115 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:47.852 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:47.855 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:47.855 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:47.856 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:47.856 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:47.857 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:47.857 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:47.858 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:47.859 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:47.859 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:47.860 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:47.860 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:47.860 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:47.861 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:51.717 7f5f031cd680 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:15:51.855 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:51.857 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:51.858 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:51.858 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:51.858 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:51.858 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:51.859 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:51.859 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:51.860 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:51.860 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:51.860 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:51.860 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:51.861 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:51.861 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:55.860 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:55.863 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:55.863 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:55.864 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:55.864 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:55.864 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:55.864 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:55.865 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:55.866 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:55.866 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:55.866 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:55.866 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:55.867 7f5f031cd680 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:55.867 7f5f031cd680 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:59.863 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:15:59.866 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:15:59.866 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:15:59.867 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:59.867 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:15:59.867 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:15:59.867 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:15:59.868 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:15:59.869 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:59.869 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:15:59.869 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:59.869 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:15:59.870 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:15:59.870 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:03.870 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:03.872 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:03.873 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:03.873 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:03.873 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:03.873 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:03.874 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:03.874 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:03.875 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:03.875 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:03.875 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:03.876 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:03.876 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:03.876 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:05.152 7f5f030e7f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:16:07.876 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:07.879 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:07.879 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:07.880 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:07.880 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:07.881 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:07.881 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:07.882 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:07.883 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:07.883 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:07.884 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:07.884 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:07.885 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:07.885 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:11.884 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:11.886 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:11.887 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:11.887 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:11.887 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:11.888 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:11.888 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:11.889 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:11.889 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:11.890 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:11.890 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:11.890 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:11.891 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:11.891 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:15.888 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:15.890 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:15.891 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:15.891 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:15.892 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:15.892 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:15.892 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:15.893 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:15.893 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:15.894 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:15.894 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:15.894 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:15.894 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:15.895 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:18.049 7f5f030e7e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:16:19.892 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:19.895 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:19.896 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:19.896 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:19.896 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:19.897 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:19.897 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:19.898 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:19.898 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:19.898 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:19.899 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:19.899 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:19.899 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:19.899 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:23.898 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:23.901 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:23.901 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:23.901 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:23.902 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:23.902 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:23.902 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:23.903 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:23.903 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:23.904 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:23.904 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:23.904 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:23.904 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:23.905 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:27.904 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:27.907 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:27.907 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:27.907 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:27.908 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:27.908 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:27.908 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:27.909 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:27.910 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:27.910 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:27.910 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:27.911 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:27.911 7f5f030e7e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:27.911 7f5f030e7e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:30.981 7f5f030e7f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:16:31.908 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:31.911 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:31.911 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:31.912 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:31.912 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:31.912 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:31.913 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:31.914 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:31.914 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:31.914 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:31.915 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:31.915 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:31.915 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:31.916 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:35.911 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:35.914 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:35.915 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:35.915 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:35.915 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:35.916 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:35.916 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:35.917 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:35.917 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:35.918 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:35.918 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:35.919 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:35.919 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:35.919 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:39.635 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:16:39.914 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:39.917 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:39.917 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:39.918 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:39.918 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:39.918 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:39.919 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:39.919 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:39.920 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:39.920 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:39.921 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:39.921 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:39.921 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:39.922 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:40.456 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:40.457 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19644088 disk_space_available_1k = 61446396 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:16:40.457 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:40.867 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:16:42.129 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:16:43.918 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:43.920 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:43.920 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:43.920 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:43.920 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:43.920 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:43.920 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:43.921 7f5f031e77c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:44.011 7f5f031e77c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:16:45.185 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19644176, partition_space_available_1k=61446308, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1292976, cpu_share=0.05, mem_res=1283992, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46812, cpu_share=0.04, mem_res=43176, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9409992, free=19648048, buffers=721896, cached=2885184, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.92, five_min_avg=1.68, fifteen_min_avg=1.78, cpu_share=1.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:16:45.186 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:45.187 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19644176 partition_space_available_1k = 61446308 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1292976 cpu_share = 0.05 mem_res = 1283992 >>contrail-database-nodemgr : << mem_virt = 46812 cpu_share = 0.04 mem_res = 43176 >> } system_mem_usage = << total = 32665120 used = 9409992 free = 19648048 buffers = 721896 cached = 2885184 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.92 five_min_avg = 1.68 fifteen_min_avg = 1.78 cpu_share = 1.75 node_type = database-node >> >> 09/01/2025 02:16:45.187 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:47.923 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:47.925 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:47.926 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:47.926 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:47.926 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:47.927 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:47.927 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:47.928 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:47.928 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:47.928 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:47.929 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:47.929 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:47.929 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:47.929 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:51.928 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:51.930 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:51.931 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:51.931 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:51.932 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:51.932 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:51.932 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:51.933 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:51.933 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:51.934 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:51.934 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:51.934 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:51.934 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:51.935 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:55.932 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:55.934 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:55.935 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:55.935 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:55.936 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:55.936 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:55.936 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:55.937 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:55.937 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:55.938 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:55.938 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:55.938 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:55.938 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:55.939 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:56.972 7f5f030e7f40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:16:59.938 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:16:59.940 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:16:59.941 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:16:59.941 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:59.941 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:16:59.942 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:16:59.942 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:16:59.943 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:16:59.943 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:59.944 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:16:59.944 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:59.944 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:16:59.945 7f5f030e7f40 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:16:59.945 7f5f030e7f40 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:03.944 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:03.947 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:03.947 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:03.947 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:03.948 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:03.948 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:03.949 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:03.949 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:03.950 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:03.950 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:03.950 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:03.950 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:03.951 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:03.951 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:07.948 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:07.951 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:07.951 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:07.951 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:07.952 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:07.952 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:07.953 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:07.954 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:07.954 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:07.954 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:07.955 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:07.955 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:07.955 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:07.955 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:10.429 7f5f033af0e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:17:11.952 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:11.955 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:11.955 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:11.956 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:11.956 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:11.956 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:11.956 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:11.957 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:11.958 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:11.958 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:11.958 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:11.958 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:11.959 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:11.959 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:13.419 7f5f033af0e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:17:15.957 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:15.959 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:15.960 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:15.960 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:15.960 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:15.961 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:15.961 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:15.963 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:15.964 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:15.965 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:15.965 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:15.965 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:15.966 7f5f033af0e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:15.966 7f5f033af0e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:16.340 7f5f031e7180 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:17:19.960 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:19.962 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:19.963 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:19.963 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:19.964 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:19.964 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:19.964 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:19.965 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:19.965 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:19.966 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:19.966 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:19.966 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:19.966 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:19.967 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:23.967 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:23.969 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:23.970 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:23.970 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:23.970 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:23.971 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:23.971 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:23.972 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:23.972 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:23.973 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:23.973 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:23.973 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:23.973 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:23.974 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:27.974 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:27.977 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:27.978 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:27.978 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:27.979 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:27.979 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:27.979 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:27.980 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:27.981 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:27.981 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:27.982 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:27.982 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:27.982 7f5f031e7180 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:27.983 7f5f031e7180 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:31.978 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:31.981 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:31.981 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:31.982 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:31.982 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:31.982 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:31.983 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:31.984 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:31.984 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:31.984 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:31.985 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:31.985 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:31.985 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:31.986 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:35.985 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:35.986 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:35.987 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:35.987 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:35.987 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:35.987 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:35.987 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:35.988 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:35.988 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:35.988 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:35.988 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:35.989 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:35.989 7f5f031cd5e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:35.989 7f5f031cd5e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:39.565 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/01/2025 02:17:39.994 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:40.001 7f5f031e7220 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:40.002 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:40.002 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:40.003 7f5f031e7220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:40.003 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:40.003 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:40.004 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:40.004 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:40.005 7f5f031e7220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:40.005 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:40.005 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:40.005 7f5f031e7220 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:40.005 7f5f031e7220 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:40.345 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:40.345 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. database_usage = [ << disk_space_used_1k = 19655608 disk_space_available_1k = 61434876 analytics_db_size_1k = 988 >>, ] >> 09/01/2025 02:17:40.346 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:40.711 7f5f05f7b940 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/01/2025 02:17:41.885 7f5f05f7b940 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/01/2025 02:17:44.003 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:44.007 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:44.007 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:44.008 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:44.008 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:44.008 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:44.009 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:44.009 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:44.010 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:44.010 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:44.010 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:44.011 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:44.011 7f5f030e7ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:44.011 7f5f030e7ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:44.931 7f5f05f7b940 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3993-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19655652, partition_space_available_1k=61434832, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1292972, cpu_share=0.05, mem_res=1283992, _EventManager__key='cassandra'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=46900, cpu_share=0.04, mem_res=43176, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9407588, free=19625004, buffers=721932, cached=2910596, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.85, five_min_avg=1.73, fifteen_min_avg=1.79, cpu_share=2.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/01/2025 02:17:44.931 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:44.932 7f5f031cdea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 19655652 partition_space_available_1k = 61434832 percentage_partition_space_used = 24 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1292972 cpu_share = 0.05 mem_res = 1283992 >>contrail-database-nodemgr : << mem_virt = 46900 cpu_share = 0.04 mem_res = 43176 >> } system_mem_usage = << total = 32665120 used = 9407588 free = 19625004 buffers = 721932 cached = 2910596 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.85 five_min_avg = 1.73 fifteen_min_avg = 1.79 cpu_share = 2.0 node_type = database-node >> >> 09/01/2025 02:17:44.932 7f5f031cdea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:47.510 7f5f030e7d60 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/01/2025 02:17:48.008 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:48.011 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:48.011 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:48.011 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:48.012 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:48.012 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:48.012 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:48.013 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:48.013 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:48.014 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:48.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:48.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:48.015 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:48.015 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:52.012 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:52.013 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:52.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:52.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:52.014 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:52.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:52.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:52.014 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:52.015 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:52.015 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:52.015 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:52.015 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:52.015 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:52.015 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:56.016 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:17:56.018 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:17:56.018 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:17:56.018 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:56.019 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:17:56.019 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:17:56.019 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:17:56.019 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:17:56.019 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:56.020 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:17:56.020 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:56.020 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:17:56.020 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:17:56.020 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:00.020 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:18:00.022 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:18:00.023 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:18:00.023 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:00.023 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:18:00.023 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:18:00.023 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:18:00.024 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:18:00.024 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:00.025 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.56:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:18:00.025 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:00.025 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:18:00.025 7f5f030e7d60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-3993-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1756691978608362 collector_name = collector_ip = 10.0.0.56:8086 collector_list = [ 10.0.0.56:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/01/2025 02:18:00.025 7f5f030e7d60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]